-- Logs begin at Wed 2022-07-27 03:15:10 UTC, end at Wed 2022-07-27 04:41:22 UTC. -- Jul 27 04:07:36.027410 dv-cinder-850989-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 27 04:07:36.035308 dv-cinder-850989-2-iscsi devstack@keystone.service[80230]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 27 04:07:36.037387 dv-cinder-850989-2-iscsi devstack@keystone.service[80230]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jul 27 04:07:36 2022] *** Jul 27 04:07:36.037387 dv-cinder-850989-2-iscsi devstack@keystone.service[80230]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 27 04:07:36.037387 dv-cinder-850989-2-iscsi devstack@keystone.service[80230]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 27 04:07:36.037387 dv-cinder-850989-2-iscsi devstack@keystone.service[80230]: nodename: dv-cinder-850989-2-iscsi Jul 27 04:07:36.037387 dv-cinder-850989-2-iscsi devstack@keystone.service[80230]: machine: x86_64 Jul 27 04:07:36.037387 dv-cinder-850989-2-iscsi devstack@keystone.service[80230]: clock source: unix Jul 27 04:07:36.037387 dv-cinder-850989-2-iscsi devstack@keystone.service[80230]: pcre jit disabled Jul 27 04:07:36.037819 dv-cinder-850989-2-iscsi devstack@keystone.service[80230]: detected number of CPU cores: 4 Jul 27 04:07:36.037819 dv-cinder-850989-2-iscsi devstack@keystone.service[80230]: current working directory: / Jul 27 04:07:36.037819 dv-cinder-850989-2-iscsi devstack@keystone.service[80230]: detected binary path: /usr/bin/uwsgi-core Jul 27 04:07:36.037819 dv-cinder-850989-2-iscsi devstack@keystone.service[80230]: your processes number limit is 47680 Jul 27 04:07:36.037819 dv-cinder-850989-2-iscsi devstack@keystone.service[80230]: your memory page size is 4096 bytes Jul 27 04:07:36.037819 dv-cinder-850989-2-iscsi devstack@keystone.service[80230]: detected max file descriptor number: 2048 Jul 27 04:07:36.037819 dv-cinder-850989-2-iscsi devstack@keystone.service[80230]: lock engine: pthread robust mutexes Jul 27 04:07:36.037819 dv-cinder-850989-2-iscsi devstack@keystone.service[80230]: thunder lock: enabled Jul 27 04:07:36.037819 dv-cinder-850989-2-iscsi devstack@keystone.service[80230]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 27 04:07:36.037819 dv-cinder-850989-2-iscsi devstack@keystone.service[80230]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 27 04:07:36.077754 dv-cinder-850989-2-iscsi devstack@keystone.service[80230]: Python main interpreter initialized at 0x560c76a749c0 Jul 27 04:07:36.077754 dv-cinder-850989-2-iscsi devstack@keystone.service[80230]: python threads support enabled Jul 27 04:07:36.077754 dv-cinder-850989-2-iscsi devstack@keystone.service[80230]: your server socket listen backlog is limited to 100 connections Jul 27 04:07:36.077754 dv-cinder-850989-2-iscsi devstack@keystone.service[80230]: your mercy for graceful operations on workers is 90 seconds Jul 27 04:07:36.078145 dv-cinder-850989-2-iscsi devstack@keystone.service[80230]: mapped 403077 bytes (393 KB) for 2 cores Jul 27 04:07:36.078255 dv-cinder-850989-2-iscsi devstack@keystone.service[80230]: *** Operational MODE: preforking *** Jul 27 04:07:36.078531 dv-cinder-850989-2-iscsi devstack@keystone.service[80230]: *** uWSGI is running in multiple interpreter mode *** Jul 27 04:07:36.078531 dv-cinder-850989-2-iscsi devstack@keystone.service[80230]: spawned uWSGI master process (pid: 80230) Jul 27 04:07:36.078733 dv-cinder-850989-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 27 04:07:36.082001 dv-cinder-850989-2-iscsi devstack@keystone.service[80230]: spawned uWSGI worker 1 (pid: 80239, cores: 1) Jul 27 04:07:36.082001 dv-cinder-850989-2-iscsi devstack@keystone.service[80230]: spawned uWSGI worker 2 (pid: 80240, cores: 1) Jul 27 04:07:36.082001 dv-cinder-850989-2-iscsi devstack@keystone.service[80230]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 27 04:07:41.575558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 27 04:07:41.575558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: from cryptography.utils import int_from_bytes Jul 27 04:07:41.576339 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 27 04:07:41.576339 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: from cryptography.utils import int_from_bytes Jul 27 04:07:41.576768 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 27 04:07:41.576768 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: from cryptography.utils import int_from_bytes Jul 27 04:07:41.577219 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 27 04:07:41.577219 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: from cryptography.utils import int_from_bytes Jul 27 04:07:41.626362 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 27 04:07:41.626546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 27 04:07:41.626735 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] command line args: [] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 27 04:07:41.626866 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 27 04:07:41.627013 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 27 04:07:41.627102 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 27 04:07:41.627182 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ================================================================================ {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 27 04:07:41.627253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] command line args: [] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 27 04:07:41.627444 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] admin_token = **** {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.627530 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 27 04:07:41.627686 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ================================================================================ {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 27 04:07:41.627776 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] config_dir = [] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.627879 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.628120 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] admin_token = **** {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.628208 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] config_source = [] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.628279 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] config_dir = [] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.628393 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.628481 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] debug = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.628625 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.628925 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] config_source = [] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.628925 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] debug = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.629128 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.629232 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.629360 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.629456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.629564 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.629668 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.629765 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.629864 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.629965 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] list_limit = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.630110 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.630208 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] log_config_append = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.630310 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] list_limit = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.630412 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.630412 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] log_dir = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.630609 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] log_config_append = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.630711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] log_file = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.630806 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.630806 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] log_dir = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.631018 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.631135 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] log_file = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.631251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.631326 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.631441 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.631524 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.631601 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.631721 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.631795 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.631795 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.631977 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.631977 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.632171 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.632304 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.632379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.632494 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.632579 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.632657 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.632773 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.632853 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.632924 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.633087 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.633171 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.633331 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.633423 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.633546 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.633625 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.633806 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.633900 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.634092 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.634367 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.46/identity {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.634487 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.634621 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] publish_errors = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.634776 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.634886 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.46/identity {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.634998 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.635117 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] publish_errors = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.635231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.635342 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.635453 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.635570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.635688 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.635820 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.635942 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] standard_threads = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.636061 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.636186 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.636365 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.636482 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.636600 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] standard_threads = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.636713 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.636826 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.636938 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] use_journal = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.636938 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] use_json = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.637161 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.637280 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] use_stderr = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.637402 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.637521 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] use_syslog = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.637635 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] use_journal = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.637635 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] use_json = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.637853 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.637993 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] use_stderr = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.638109 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.638222 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] use_syslog = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.638324 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.638446 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 04:07:41.638567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.638680 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.638793 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.638905 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.639018 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.639138 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.639258 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.639370 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.639483 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.639600 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.639713 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.639833 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.639947 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.640060 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.640178 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.640291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.640403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.640523 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.640636 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.640747 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.640859 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.640971 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.641095 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.641224 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.641339 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.641455 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.641568 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.641688 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.641807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.642012 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.642141 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.642256 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.642369 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.642481 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.642593 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.642717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.642837 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.642950 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.643062 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.643193 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.643306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.643462 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.643604 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.643719 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.643830 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.643942 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.644053 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.644173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.644293 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.644293 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.644512 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.644624 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.644624 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.644895 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.645048 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.645198 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.645353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.645488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.645612 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.645732 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.645846 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.645959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.646102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.646217 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.646348 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.646463 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.646575 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.646686 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.646798 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.646916 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.647038 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.647152 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.647264 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.647382 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.647494 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.647494 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.647779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.647895 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.648014 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.648133 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.648252 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.648366 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.648487 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.648601 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.648713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.648824 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.648936 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.649055 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.649179 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.649296 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.649415 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.649530 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.649642 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.649753 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.649873 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.650016 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.650119 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.650214 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.650308 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.650412 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.650518 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.650620 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.650716 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.650811 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.650906 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.651041 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.651171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.651286 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.651399 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.651512 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.651626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.651745 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.651866 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.651981 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.652094 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.652206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.652319 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.652440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.652565 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.652686 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.652802 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.652925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.653046 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.653150 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.653268 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.653393 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.653497 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.653497 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.653719 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.653834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.653947 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.654076 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.654194 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.654308 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.654422 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.654525 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.654626 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.654745 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.654860 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.654981 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.655096 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.655208 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.655321 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.655444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.655564 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.655678 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.655793 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.655905 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.656023 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.656145 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.656260 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.656372 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.656485 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.656598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.656711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.656832 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.656952 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.657067 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.657190 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.657190 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.657403 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.657403 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.657624 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.657744 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.657859 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.658011 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.658128 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.658256 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.658372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.658486 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.658598 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.658598 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.658809 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.659002 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.659119 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.659842 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.659961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.660080 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.660194 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.660314 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.660437 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.660562 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.660677 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.660795 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.660917 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.661032 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.661145 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.661145 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.661365 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.661477 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.661589 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.661707 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.661821 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.661942 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.662073 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.662197 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.662312 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.662426 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.662547 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.662661 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.662774 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.662888 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.663001 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] auth.external = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.663121 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.663248 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.663364 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.663485 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.663601 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.663720 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.663834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.663961 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] auth.password = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.664102 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] auth.external = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.664231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] auth.token = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.664347 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.664462 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.664576 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.664697 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.664818 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.664941 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.665057 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] auth.password = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.665173 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.665287 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] auth.token = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.665402 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.665516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.665638 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.665754 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.665876 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.666033 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.666149 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] credential.caching = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.666264 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.666391 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.666507 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.666622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.666737 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.666851 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.666965 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.667088 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.667204 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] credential.caching = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.667319 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.667439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.667555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.667673 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.667796 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.667912 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.668032 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.668147 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.668261 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.668377 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.668498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.668615 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.668729 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.668852 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.668968 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.669082 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.669204 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.669320 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.669436 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.669550 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.669664 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.669778 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.669900 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] federation.caching = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.670029 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.670146 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.670259 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.670373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.670488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.670614 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.670614 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.670829 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.670943 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.671057 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] federation.caching = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.671172 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.671294 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.671410 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.671525 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.671639 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.671752 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.671866 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.671988 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.672104 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.672218 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.672218 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.672431 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.672431 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity.caching = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.672656 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.672772 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.672887 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.673001 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.673115 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.673318 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.673436 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.673552 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.673666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.673781 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.673895 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity.caching = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.674061 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.674180 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.674295 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.674409 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.674523 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.674637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.674759 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.675696 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.675818 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.675934 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.676049 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.676163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.676287 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.676404 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.676526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.676647 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.676764 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.676764 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.676986 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.676986 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.677202 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.677317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.677431 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.677545 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.677667 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.677784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.677899 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.678027 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.678143 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.678258 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.678389 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.678506 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.678621 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.678735 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.678850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.678850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.679071 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.679193 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.679309 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.679423 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.679538 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.679651 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.679773 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.679890 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.680004 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.680119 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.680233 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.680347 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.680469 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.680585 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.680700 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.680820 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.680936 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.681052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.681174 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.681296 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.681411 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.681534 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.681650 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.681765 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.681888 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.682021 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.682137 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.682252 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.682366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.682481 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.682612 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.682735 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.682851 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.682964 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.683080 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.683194 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.683317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.683433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.683547 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.683661 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.683775 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.683889 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.684011 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.684128 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.684254 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.684370 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.684485 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.684600 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.684722 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.684838 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.684952 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.685067 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.685182 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.685304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.685420 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.685535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.685649 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.685763 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.685878 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.686047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.686168 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.686282 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.686397 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.686511 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.686625 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.686748 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.686864 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.686979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.687093 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.687209 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.687323 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.687446 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.687562 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.687677 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.687793 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.688155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.688275 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.688399 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.688514 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.688629 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.688743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.688857 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.688972 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.689094 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.689210 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.689326 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.689440 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.689555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.689669 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.689792 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.689908 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.690036 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.690153 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.690267 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.690382 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.690604 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.690784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.690968 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.691140 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.691306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.691470 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.691640 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.691804 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.691924 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.692039 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.692153 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.692267 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.692390 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.692507 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.692622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.692736 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.692850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.692850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.693079 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.693195 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.693309 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.693422 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.693537 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.693651 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.693772 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.693888 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.694022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.694139 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.694254 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.694368 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.694495 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.694612 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] resource.caching = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.694727 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.694848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.694963 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.695078 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.695200 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.695316 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.695430 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.695544 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.695658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.695772 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.695893 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.696009 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.696123 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.696237 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.696351 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] resource.caching = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.696465 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.696587 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.696703 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.696817 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.696930 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] role.caching = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.697044 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.697164 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] role.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.697356 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.697474 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.697597 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.697713 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.697832 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.697947 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.698097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.698215 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.698329 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.698444 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.698558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.698672 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.698796 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] role.caching = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.698911 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.699026 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] role.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.699140 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.699254 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.699377 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.699493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.699607 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.699722 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.699837 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.699951 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.700073 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.700189 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.700304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.700419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.700534 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.700648 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.700771 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.700888 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.701003 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.701117 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.701231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.701345 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.701467 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.701582 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.701697 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.701810 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.701925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.702092 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.702220 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.702337 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.702451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.702565 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.702679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.702792 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.702914 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.703030 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.703145 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.703145 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.703357 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.703471 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.703593 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.703708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.703822 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.703936 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.704051 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.704173 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.704289 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.704403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.704517 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.704631 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.704745 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.704866 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.704982 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.705135 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] token.caching = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.705257 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.705374 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.705488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.705611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.705727 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.705841 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.705955 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.706149 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.706264 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.706390 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.706507 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.706755 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.706872 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.706986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.707100 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.707224 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.707340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] token.caching = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.707455 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.707569 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.707683 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.707797 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.707920 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.708036 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.708157 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.708272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.708386 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.708500 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.708622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.708738 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.708852 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.708966 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.709086 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.709238 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.709364 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.709480 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.709595 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.connection = **** {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.709718 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.709847 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.710007 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.710163 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.710293 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.710418 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.710534 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.710648 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.710768 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.710884 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.711007 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.711123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.711237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.711352 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.711352 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.711555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.711676 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.711792 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.connection = **** {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.711915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.712039 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.712156 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.712270 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.712385 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.712499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.712622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.712738 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.712852 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.712965 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.713079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.713194 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.713316 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.713433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.713547 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.713661 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.713775 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.713889 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.714076 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 27 04:07:41.714197 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.714197 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.714197 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.714197 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.714197 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.714197 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.714197 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.714197 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.715027 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 04:07:41.715027 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80240) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 27 04:07:41.742832 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.743060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.743287 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.747620 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.747860 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.748091 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.751643 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.756703 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.781091 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.781319 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.784901 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.785134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.835377 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.835643 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.886133 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.886364 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.887077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.887319 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.952346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.952493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.952577 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.952673 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.952780 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.952871 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.952999 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.953083 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.953219 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.953311 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.953437 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.953521 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.977535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.977643 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.977790 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.977873 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.977947 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.978090 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.978169 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.978257 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.978373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.978456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.978590 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.978672 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.978807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.978899 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.990538 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:41.990725 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:42.003390 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:42.003715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:42.003816 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:42.003952 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:42.004036 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:42.004179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:42.004260 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:42.004404 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:42.004487 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:42.004624 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:42.004709 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:42.004843 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:42.004925 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80240) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:42.005058 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 04:07:42.022521 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.022735 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 27 04:07:42.022954 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.023077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.023173 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 27 04:07:42.023281 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 27 04:07:42.023488 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.023568 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.023691 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 27 04:07:42.023807 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.023962 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.024069 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.024223 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.024328 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.024480 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.024733 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.033987 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.034336 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.034428 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.034609 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.034832 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.034917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.034991 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.035153 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.041752 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.041938 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.046477 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.046698 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.046779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.046978 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.047058 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.047262 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.047344 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.047542 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.047624 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.047822 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.047912 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.048082 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.048163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.048350 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.048431 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.048621 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.048709 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.048887 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.048968 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.049157 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.049236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.049420 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.072882 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.073095 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.074387 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.074600 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.074696 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.074880 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.079325 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.079445 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.079609 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.079723 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.079896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.080003 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.087064 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.087156 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.087347 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.087445 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.091412 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.091700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.091993 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.092277 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.092552 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.092725 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.092841 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.093025 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.093106 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.093314 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.093402 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.093602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.093885 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.094208 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.094496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.094781 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.109240 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.109520 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.110630 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.110914 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.113933 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.115448 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.117062 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.117358 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.118638 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.118941 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.121693 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.123327 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.124889 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.126532 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.128087 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.128386 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.128751 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.129051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.129339 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.129626 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.129991 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.130296 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.130585 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.130875 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.131159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.131450 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.146039 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.146330 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.146600 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.146872 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.147911 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.148205 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.148480 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.148751 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.153664 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.155231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.155506 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.155649 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.157197 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.157478 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.158408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.158688 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.158990 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.159430 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.159701 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.160407 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.160690 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.160986 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.161272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.161566 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.171037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.171319 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.171600 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.171881 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.172157 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.172428 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.172708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.172964 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.173246 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.173530 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.173816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.174143 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.174425 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.174703 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.188906 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.191155 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.191931 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.194163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.194869 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.197114 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.197273 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.198972 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.199256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.199537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.199618 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.201086 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.201392 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.201679 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.205895 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.207628 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.208181 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.209797 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.210535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.210824 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.211107 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.211382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.212744 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.213036 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.213325 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.213608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.219354 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.219636 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.219915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.222177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.222572 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.222874 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.227189 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.227469 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.227799 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.228042 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.228315 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.228596 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.228865 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.229138 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.229515 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.229716 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80240) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 04:07:42.229799 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.230019 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.230104 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.230311 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.230397 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.230606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.230697 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.230866 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.230950 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.231154 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.231430 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.231699 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.232073 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.232359 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.232656 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.232934 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.233203 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.233467 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80240) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 04:07:42.264674 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x560c76a749c0 pid: 80239 (default app) Jul 27 04:07:42.267506 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x560c76a749c0 pid: 80240 (default app) Jul 27 04:07:42.267944 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a8519-e7a0-4fa6-b178-6e63a276ff4f None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:42.268236 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a8519-e7a0-4fa6-b178-6e63a276ff4f None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:42.268512 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a8519-e7a0-4fa6-b178-6e63a276ff4f None None] PATH_INFO: `/v3` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:42.272921 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1/1] 10.222.0.46 () {50 vars in 791 bytes} [Wed Jul 27 04:07:42 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:07:49.880506 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341b117-84e5-438e-8cfd-6d1563fb8dd2 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:49.881050 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341b117-84e5-438e-8cfd-6d1563fb8dd2 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:49.881050 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341b117-84e5-438e-8cfd-6d1563fb8dd2 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:49.885453 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1/2] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:07:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 04:07:49.890972 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414df49-1ca3-498a-9799-1986e0ac5cb0 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:49.891354 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414df49-1ca3-498a-9799-1986e0ac5cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:49.891521 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414df49-1ca3-498a-9799-1986e0ac5cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:49.962498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: INFO dbcounter [None req-c414df49-1ca3-498a-9799-1986e0ac5cb0 None None] Registered counter for database keystone Jul 27 04:07:49.983059 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG oslo_db.sqlalchemy.engines [None req-c414df49-1ca3-498a-9799-1986e0ac5cb0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80239) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 27 04:07:49.989165 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writer thread running {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 27 04:07:50.165915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG passlib.handlers.bcrypt [None req-c414df49-1ca3-498a-9799-1986e0ac5cb0 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80239) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 27 04:07:50.166339 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG passlib.handlers.bcrypt [None req-c414df49-1ca3-498a-9799-1986e0ac5cb0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80239) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 27 04:07:50.219925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-c414df49-1ca3-498a-9799-1986e0ac5cb0 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:07:50.222178 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c414df49-1ca3-498a-9799-1986e0ac5cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:07:50.344660 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 2/3] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:07:49 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 27 04:07:50.474675 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341b117-84e5-438e-8cfd-6d1563fb8dd2 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:07:50.543694 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: INFO dbcounter [None req-6341b117-84e5-438e-8cfd-6d1563fb8dd2 None None] Registered counter for database keystone Jul 27 04:07:50.562763 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG oslo_db.sqlalchemy.engines [None req-6341b117-84e5-438e-8cfd-6d1563fb8dd2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80240) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 27 04:07:50.570576 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writer thread running {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 27 04:07:50.690839 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341b117-84e5-438e-8cfd-6d1563fb8dd2 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:07:50.695297 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da634504-5980-4a29-bed6-d4f1ed4cd66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:07:50.695945 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da634504-5980-4a29-bed6-d4f1ed4cd66a admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:50.696237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da634504-5980-4a29-bed6-d4f1ed4cd66a admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:50.696512 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da634504-5980-4a29-bed6-d4f1ed4cd66a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:50.704684 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da634504-5980-4a29-bed6-d4f1ed4cd66a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:07:50.756884 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: WARNING keystone.common.rbac_enforcer.enforcer [None req-da634504-5980-4a29-bed6-d4f1ed4cd66a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 27 04:07:50.757172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da634504-5980-4a29-bed6-d4f1ed4cd66a admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:07:50.767033 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 2/4] 10.222.0.46 () {58 vars in 1324 bytes} [Wed Jul 27 04:07:50 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 04:07:50.773088 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414df49-1ca3-498a-9799-1986e0ac5cb0 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:07:50.787879 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414df49-1ca3-498a-9799-1986e0ac5cb0 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:07:50.792326 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981017f-b69c-485f-bb2c-eaa3751c445b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:07:50.793443 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981017f-b69c-485f-bb2c-eaa3751c445b admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:50.793731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981017f-b69c-485f-bb2c-eaa3751c445b admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:50.794040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981017f-b69c-485f-bb2c-eaa3751c445b admin admin] PATH_INFO: `/v3/projects/5672fa915b944f15a9f6ebcb29dba655` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:50.795562 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f981017f-b69c-485f-bb2c-eaa3751c445b admin admin] RBAC: Authorizing `identity:get_project(project_id=5672fa915b944f15a9f6ebcb29dba655)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:07:50.846624 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f981017f-b69c-485f-bb2c-eaa3751c445b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 27 04:07:50.846974 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f981017f-b69c-485f-bb2c-eaa3751c445b admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:07:50.851580 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 3/5] 10.222.0.46 () {56 vars in 1207 bytes} [Wed Jul 27 04:07:50 2022] GET /identity/v3/projects/5672fa915b944f15a9f6ebcb29dba655 => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:07:52.820529 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb9781-58dd-4d3e-b109-46cbbc4ca4b1 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:52.821021 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb9781-58dd-4d3e-b109-46cbbc4ca4b1 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:52.821021 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb9781-58dd-4d3e-b109-46cbbc4ca4b1 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:52.825134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 3/6] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:07:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:07:52.830382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4d9be-4624-4bd8-81e5-ce5781b637e7 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:52.830766 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4d9be-4624-4bd8-81e5-ce5781b637e7 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:52.831504 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4d9be-4624-4bd8-81e5-ce5781b637e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:52.851201 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-b9c4d9be-4624-4bd8-81e5-ce5781b637e7 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:07:52.852721 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b9c4d9be-4624-4bd8-81e5-ce5781b637e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:07:52.860555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 4/7] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:07:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 27 04:07:52.882357 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb9781-58dd-4d3e-b109-46cbbc4ca4b1 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:07:52.894940 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb9781-58dd-4d3e-b109-46cbbc4ca4b1 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:07:52.899140 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41039077-e4bd-4f76-8cc0-75e8b9fdd857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:07:52.899745 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41039077-e4bd-4f76-8cc0-75e8b9fdd857 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:52.900029 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41039077-e4bd-4f76-8cc0-75e8b9fdd857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:52.900306 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41039077-e4bd-4f76-8cc0-75e8b9fdd857 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:52.907965 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41039077-e4bd-4f76-8cc0-75e8b9fdd857 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:07:52.909738 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41039077-e4bd-4f76-8cc0-75e8b9fdd857 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:07:52.918434 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 4/8] 10.222.0.46 () {58 vars in 1324 bytes} [Wed Jul 27 04:07:52 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 04:07:52.923638 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4d9be-4624-4bd8-81e5-ce5781b637e7 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:07:52.932512 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4d9be-4624-4bd8-81e5-ce5781b637e7 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:07:52.936852 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612917d-40a2-434c-aaad-9a460abb5ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:07:52.937487 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612917d-40a2-434c-aaad-9a460abb5ee7 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:52.937772 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612917d-40a2-434c-aaad-9a460abb5ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:52.938341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612917d-40a2-434c-aaad-9a460abb5ee7 admin admin] PATH_INFO: `/v3/users/32924f9a30fb4e7b835bb29d116319d9` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:52.939716 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2612917d-40a2-434c-aaad-9a460abb5ee7 admin admin] RBAC: Authorizing `identity:get_user(user_id=32924f9a30fb4e7b835bb29d116319d9)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:07:52.942061 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2612917d-40a2-434c-aaad-9a460abb5ee7 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:07:52.943597 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 5/9] 10.222.0.46 () {56 vars in 1198 bytes} [Wed Jul 27 04:07:52 2022] GET /identity/v3/users/32924f9a30fb4e7b835bb29d116319d9 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:07:54.976615 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec6312-23be-4fc1-9e15-fc2bfaaff5fb None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:54.977123 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec6312-23be-4fc1-9e15-fc2bfaaff5fb None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:54.977291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec6312-23be-4fc1-9e15-fc2bfaaff5fb None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:54.978674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 5/10] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:07:54.983760 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58159570-f93d-4ba5-9529-0248db56b53d None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:54.984142 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58159570-f93d-4ba5-9529-0248db56b53d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:54.984327 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58159570-f93d-4ba5-9529-0248db56b53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:55.004001 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-58159570-f93d-4ba5-9529-0248db56b53d None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:07:55.005200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-58159570-f93d-4ba5-9529-0248db56b53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:07:55.058488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c79fc-ac4b-43e6-bd69-448f79b4eac1 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:55.058870 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c79fc-ac4b-43e6-bd69-448f79b4eac1 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:55.059216 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c79fc-ac4b-43e6-bd69-448f79b4eac1 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:55.060620 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 6/11] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 04:07:55.066337 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173e44d-a89e-4a05-901a-c105907a8d52 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:55.066624 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173e44d-a89e-4a05-901a-c105907a8d52 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:55.066902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173e44d-a89e-4a05-901a-c105907a8d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:55.085271 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 6/12] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:07:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:07:55.108190 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58159570-f93d-4ba5-9529-0248db56b53d None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:07:55.109248 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG passlib.handlers.bcrypt [None req-a173e44d-a89e-4a05-901a-c105907a8d52 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80240) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 27 04:07:55.109685 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG passlib.handlers.bcrypt [None req-a173e44d-a89e-4a05-901a-c105907a8d52 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80240) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 27 04:07:55.131614 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:07:55.155393 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-a173e44d-a89e-4a05-901a-c105907a8d52 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:07:55.157003 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-a173e44d-a89e-4a05-901a-c105907a8d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:07:55.209469 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58159570-f93d-4ba5-9529-0248db56b53d None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:07:55.242501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59547e-1210-4eaf-b705-a940c05928d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:07:55.243089 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59547e-1210-4eaf-b705-a940c05928d0 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:55.243379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59547e-1210-4eaf-b705-a940c05928d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:55.243659 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59547e-1210-4eaf-b705-a940c05928d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:55.255179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b59547e-1210-4eaf-b705-a940c05928d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:07:55.258687 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 7/13] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:07:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 04:07:55.263986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-4b59547e-1210-4eaf-b705-a940c05928d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:07:55.263986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:07:55.263986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:07:55.263986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:07:55.263986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:07:55.263986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:07:55.263986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:07:55.263986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:07:55.263986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:07:55.263986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:07:55.263986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:07:55.263986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:07:55.263986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:07:55.263986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:07:55.263986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:07:55.263986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:07:55.265046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:07:55.265046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:07:55.265046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:07:55.265046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:07:55.265046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:07:55.265046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:07:55.265046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:07:55.265046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:07:55.265046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:07:55.265046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:07:55.265046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:07:55.265046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:07:55.265046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:07:55.265046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:07:55.265046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:07:55.265046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:07:55.266072 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:07:55.266072 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:07:55.266072 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:07:55.266072 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:07:55.266072 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:07:55.266072 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:07:55.266072 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:07:55.266072 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:07:55.266072 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:07:55.266072 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:07:55.266072 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:07:55.266072 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:07:55.266072 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 7/14] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:07:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:07:55.273443 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173e44d-a89e-4a05-901a-c105907a8d52 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:07:55.280608 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59547e-1210-4eaf-b705-a940c05928d0 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:07:55.360262 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:07:55.374490 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173e44d-a89e-4a05-901a-c105907a8d52 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:07:55.376851 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:07:55.379304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59547e-1210-4eaf-b705-a940c05928d0 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:07:55.412961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925bde4c-7176-4763-8016-f448319ef149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:07:55.413335 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a726b3a-bda9-431e-9cc5-837e8b6ae0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:07:55.413962 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a726b3a-bda9-431e-9cc5-837e8b6ae0e2 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:55.414071 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925bde4c-7176-4763-8016-f448319ef149 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:55.414272 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a726b3a-bda9-431e-9cc5-837e8b6ae0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:55.414478 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925bde4c-7176-4763-8016-f448319ef149 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:55.414577 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a726b3a-bda9-431e-9cc5-837e8b6ae0e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:55.414871 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925bde4c-7176-4763-8016-f448319ef149 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:55.415839 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925bde4c-7176-4763-8016-f448319ef149 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:07:55.416216 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925bde4c-7176-4763-8016-f448319ef149 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:07:55.417701 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925bde4c-7176-4763-8016-f448319ef149 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:07:55.430371 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 8/15] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:07:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:07:55.435781 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925bde4c-7176-4763-8016-f448319ef149 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:07:55.482114 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a726b3a-bda9-431e-9cc5-837e8b6ae0e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:07:55.483587 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a726b3a-bda9-431e-9cc5-837e8b6ae0e2 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:07:55.538672 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925bde4c-7176-4763-8016-f448319ef149 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:07:55.565509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:07:55.573281 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076fd46-bc52-448a-8a11-4abca8cb3685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:07:55.574167 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076fd46-bc52-448a-8a11-4abca8cb3685 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:55.574459 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076fd46-bc52-448a-8a11-4abca8cb3685 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:55.574739 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076fd46-bc52-448a-8a11-4abca8cb3685 None admin] PATH_INFO: `/v3/users/32924f9a30fb4e7b835bb29d116319d9` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:55.576280 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7076fd46-bc52-448a-8a11-4abca8cb3685 None admin] RBAC: Authorizing `identity:get_user(user_id=32924f9a30fb4e7b835bb29d116319d9)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:07:55.577724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7076fd46-bc52-448a-8a11-4abca8cb3685 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:07:55.579532 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 9/16] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:07:55 2022] GET /identity/v3/users/32924f9a30fb4e7b835bb29d116319d9 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:07:55.585312 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 8/17] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:07:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 04:07:55.585475 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076fd46-bc52-448a-8a11-4abca8cb3685 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:07:55.590714 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a726b3a-bda9-431e-9cc5-837e8b6ae0e2 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:07:55.619598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:07:55.700309 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076fd46-bc52-448a-8a11-4abca8cb3685 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:07:55.735323 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477ddee-9111-4c9e-a9be-989fda904551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:07:55.735978 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477ddee-9111-4c9e-a9be-989fda904551 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:55.736422 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477ddee-9111-4c9e-a9be-989fda904551 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:55.736694 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477ddee-9111-4c9e-a9be-989fda904551 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:55.738583 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477ddee-9111-4c9e-a9be-989fda904551 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:07:55.739895 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477ddee-9111-4c9e-a9be-989fda904551 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:07:55.741468 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 10/18] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:07:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:07:55.747217 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477ddee-9111-4c9e-a9be-989fda904551 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:07:55.800125 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a726b3a-bda9-431e-9cc5-837e8b6ae0e2 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:07:55.834397 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ffe5a-68d8-45e1-98e2-49ecd38d5d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:07:55.835059 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ffe5a-68d8-45e1-98e2-49ecd38d5d83 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:55.835341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ffe5a-68d8-45e1-98e2-49ecd38d5d83 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:55.835623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ffe5a-68d8-45e1-98e2-49ecd38d5d83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:55.845870 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696ffe5a-68d8-45e1-98e2-49ecd38d5d83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:07:55.847300 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477ddee-9111-4c9e-a9be-989fda904551 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:07:55.848339 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696ffe5a-68d8-45e1-98e2-49ecd38d5d83 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:07:55.848957 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:07:55.863963 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-696ffe5a-68d8-45e1-98e2-49ecd38d5d83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:07:55.863963 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:07:55.863963 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:07:55.863963 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:07:55.863963 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:07:55.863963 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:07:55.863963 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:07:55.863963 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:07:55.863963 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:07:55.863963 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:07:55.863963 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:07:55.863963 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:07:55.863963 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:07:55.863963 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:07:55.863963 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:07:55.864769 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:07:55.864769 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:07:55.864769 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:07:55.864769 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:07:55.864769 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:07:55.864769 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:07:55.864769 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:07:55.864769 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:07:55.864769 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:07:55.864769 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:07:55.864769 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:07:55.864769 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:07:55.864769 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:07:55.864769 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:07:55.864769 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:07:55.864769 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:07:55.864769 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:07:55.865549 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:07:55.865549 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:07:55.865549 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:07:55.865549 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:07:55.865549 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 9/19] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:07:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:07:55.870975 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ffe5a-68d8-45e1-98e2-49ecd38d5d83 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:07:55.883868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80289a6-bc36-47f9-bf99-4a2685c26594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:07:55.884491 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80289a6-bc36-47f9-bf99-4a2685c26594 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:55.884774 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80289a6-bc36-47f9-bf99-4a2685c26594 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:55.885053 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80289a6-bc36-47f9-bf99-4a2685c26594 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:55.885852 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80289a6-bc36-47f9-bf99-4a2685c26594 None admin] RBAC: Adding query filter params (role.id=cf771a8e94ee46149eb081347fea6d67, scope.domain.id=default, user.id=32924f9a30fb4e7b835bb29d116319d9) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:07:55.886189 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80289a6-bc36-47f9-bf99-4a2685c26594 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:07:55.887626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80289a6-bc36-47f9-bf99-4a2685c26594 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:07:55.896459 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 11/20] 10.222.0.46 () {56 vars in 1322 bytes} [Wed Jul 27 04:07:55 2022] GET /identity/v3/role_assignments?user.id=32924f9a30fb4e7b835bb29d116319d9&scope.domain.id=default&role.id=cf771a8e94ee46149eb081347fea6d67 => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:07:55.904996 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:07:55.978005 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ffe5a-68d8-45e1-98e2-49ecd38d5d83 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:07:56.013152 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b17984-36dd-48db-a07d-ada44b99ac3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:07:56.013807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b17984-36dd-48db-a07d-ada44b99ac3e None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:56.014083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b17984-36dd-48db-a07d-ada44b99ac3e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:56.014357 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b17984-36dd-48db-a07d-ada44b99ac3e None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:56.015064 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b17984-36dd-48db-a07d-ada44b99ac3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:07:56.015368 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b17984-36dd-48db-a07d-ada44b99ac3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:07:56.016778 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b17984-36dd-48db-a07d-ada44b99ac3e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:07:56.036390 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 10/21] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:07:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:07:58.030281 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386684f-dbf0-4b4c-9150-076f532beaf1 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:58.031308 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386684f-dbf0-4b4c-9150-076f532beaf1 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:58.031816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386684f-dbf0-4b4c-9150-076f532beaf1 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:58.033354 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 12/22] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:07:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:07:58.039314 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad5670-080a-42db-bcd2-970064831a06 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:58.039702 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad5670-080a-42db-bcd2-970064831a06 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:58.039867 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad5670-080a-42db-bcd2-970064831a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:58.063449 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-56ad5670-080a-42db-bcd2-970064831a06 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:07:58.064647 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-56ad5670-080a-42db-bcd2-970064831a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:07:58.163174 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 11/23] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:07:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:07:58.189336 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386684f-dbf0-4b4c-9150-076f532beaf1 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:07:58.324574 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386684f-dbf0-4b4c-9150-076f532beaf1 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:07:58.364431 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5404288f-9987-47c3-89c1-a5444af38768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:07:58.365095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404288f-9987-47c3-89c1-a5444af38768 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:58.365380 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404288f-9987-47c3-89c1-a5444af38768 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:58.365653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404288f-9987-47c3-89c1-a5444af38768 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:58.383833 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5404288f-9987-47c3-89c1-a5444af38768 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:07:58.392661 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-5404288f-9987-47c3-89c1-a5444af38768 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:07:58.392661 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:07:58.392661 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:07:58.392661 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:07:58.392661 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:07:58.392661 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:07:58.392661 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:07:58.392661 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:07:58.392661 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:07:58.392661 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:07:58.392661 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:07:58.392661 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:07:58.392661 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:07:58.392661 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:07:58.392661 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:07:58.392661 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:07:58.393794 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:07:58.393794 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:07:58.393794 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:07:58.393794 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:07:58.393794 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:07:58.393794 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:07:58.393794 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:07:58.393794 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:07:58.393794 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:07:58.393794 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:07:58.393794 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:07:58.393794 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:07:58.393794 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:07:58.393794 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:07:58.393794 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:07:58.393794 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:07:58.394573 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:07:58.394573 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:07:58.394573 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:07:58.394573 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:07:58.394573 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:07:58.394573 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:07:58.394573 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:07:58.394573 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:07:58.394573 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:07:58.394573 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:07:58.394573 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:07:58.394573 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:07:58.398638 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 13/24] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:07:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:07:58.404038 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad5670-080a-42db-bcd2-970064831a06 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:07:58.439271 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:07:58.560996 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad5670-080a-42db-bcd2-970064831a06 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:07:58.561734 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165a47d-6b57-47a5-a471-031f003d5062 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:58.562289 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165a47d-6b57-47a5-a471-031f003d5062 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:58.562834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165a47d-6b57-47a5-a471-031f003d5062 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:58.565163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 14/25] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:07:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:07:58.570094 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d53325-684d-4a88-b7b9-08f46079a1b4 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:58.570623 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d53325-684d-4a88-b7b9-08f46079a1b4 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:58.571098 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d53325-684d-4a88-b7b9-08f46079a1b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:58.603791 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-c5d53325-684d-4a88-b7b9-08f46079a1b4 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:07:58.604992 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c5d53325-684d-4a88-b7b9-08f46079a1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:07:58.628261 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d88f50-c3ae-4f74-b6ac-deaf6f9f7a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:07:58.628915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d88f50-c3ae-4f74-b6ac-deaf6f9f7a38 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:58.629205 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d88f50-c3ae-4f74-b6ac-deaf6f9f7a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:58.629481 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d88f50-c3ae-4f74-b6ac-deaf6f9f7a38 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:58.630249 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d88f50-c3ae-4f74-b6ac-deaf6f9f7a38 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:07:58.630563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d88f50-c3ae-4f74-b6ac-deaf6f9f7a38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:07:58.632029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d88f50-c3ae-4f74-b6ac-deaf6f9f7a38 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:07:58.640186 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:07:58.657339 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 12/26] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:07:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:07:58.666481 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d88f50-c3ae-4f74-b6ac-deaf6f9f7a38 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:07:58.728820 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 15/27] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:07:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:07:58.762559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d53325-684d-4a88-b7b9-08f46079a1b4 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:07:58.882412 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:07:58.919934 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d88f50-c3ae-4f74-b6ac-deaf6f9f7a38 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:07:58.975424 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d53325-684d-4a88-b7b9-08f46079a1b4 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:07:59.000322 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc445f8-7a45-47fc-94e0-e99df8164fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:07:59.000988 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc445f8-7a45-47fc-94e0-e99df8164fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:59.001273 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc445f8-7a45-47fc-94e0-e99df8164fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:59.001549 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc445f8-7a45-47fc-94e0-e99df8164fc8 None admin] PATH_INFO: `/v3/users/32924f9a30fb4e7b835bb29d116319d9` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:59.005056 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc445f8-7a45-47fc-94e0-e99df8164fc8 None admin] RBAC: Authorizing `identity:get_user(user_id=32924f9a30fb4e7b835bb29d116319d9)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:07:59.006566 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc445f8-7a45-47fc-94e0-e99df8164fc8 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:07:59.008203 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 13/28] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:07:58 2022] GET /identity/v3/users/32924f9a30fb4e7b835bb29d116319d9 => generated 247 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:07:59.011080 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8c810-b98c-4b64-85f3-32583edef213 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:59.011369 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8c810-b98c-4b64-85f3-32583edef213 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:59.011649 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8c810-b98c-4b64-85f3-32583edef213 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:59.020082 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 14/29] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:07:59 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:07:59.020959 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8c810-b98c-4b64-85f3-32583edef213 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:07:59.050182 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96541408-a994-4c8a-a51f-b16a93c24fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:07:59.050831 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96541408-a994-4c8a-a51f-b16a93c24fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:59.051329 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96541408-a994-4c8a-a51f-b16a93c24fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:59.051776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96541408-a994-4c8a-a51f-b16a93c24fb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:59.071820 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96541408-a994-4c8a-a51f-b16a93c24fb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:07:59.073515 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96541408-a994-4c8a-a51f-b16a93c24fb6 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:07:59.091597 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-96541408-a994-4c8a-a51f-b16a93c24fb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:07:59.091597 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:07:59.091597 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:07:59.091597 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:07:59.091597 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:07:59.091597 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:07:59.091597 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:07:59.091597 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:07:59.091597 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:07:59.091597 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:07:59.091597 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:07:59.091597 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:07:59.091597 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:07:59.091597 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:07:59.091597 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:07:59.092479 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:07:59.092479 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:07:59.092479 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:07:59.092479 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:07:59.092479 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:07:59.092479 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:07:59.092479 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:07:59.092479 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:07:59.092479 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:07:59.092479 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:07:59.092479 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:07:59.092479 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:07:59.092479 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:07:59.092479 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:07:59.092479 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:07:59.092479 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:07:59.092479 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:07:59.093358 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:07:59.093358 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:07:59.093358 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:07:59.093358 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:07:59.095202 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 16/30] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:07:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:07:59.096761 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e8028c-8acf-4021-99d1-f20fc52b85cb None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:59.097255 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e8028c-8acf-4021-99d1-f20fc52b85cb None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:59.097729 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e8028c-8acf-4021-99d1-f20fc52b85cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:59.149791 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8c810-b98c-4b64-85f3-32583edef213 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:07:59.190125 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faab459-5e43-4ab2-a473-ee7936cab457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:07:59.191434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faab459-5e43-4ab2-a473-ee7936cab457 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:59.191890 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faab459-5e43-4ab2-a473-ee7936cab457 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:59.192312 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faab459-5e43-4ab2-a473-ee7936cab457 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:59.193824 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faab459-5e43-4ab2-a473-ee7936cab457 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:07:59.195546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faab459-5e43-4ab2-a473-ee7936cab457 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:07:59.197824 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 15/31] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:07:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:07:59.199349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a8239-83d2-4ddf-93bf-51852a359e41 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:59.199776 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a8239-83d2-4ddf-93bf-51852a359e41 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:59.200271 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a8239-83d2-4ddf-93bf-51852a359e41 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:59.201744 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 16/32] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:07:59.207325 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a8239-83d2-4ddf-93bf-51852a359e41 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:07:59.225067 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-a2e8028c-8acf-4021-99d1-f20fc52b85cb None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:07:59.226439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-a2e8028c-8acf-4021-99d1-f20fc52b85cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:07:59.228716 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:07:59.235810 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:07:59.318771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 17/33] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:07:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:07:59.319947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e8028c-8acf-4021-99d1-f20fc52b85cb None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:07:59.339143 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a8239-83d2-4ddf-93bf-51852a359e41 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:07:59.391425 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b74d4-77df-4e13-9ff8-d38a30543753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:07:59.392577 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b74d4-77df-4e13-9ff8-d38a30543753 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:59.392963 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b74d4-77df-4e13-9ff8-d38a30543753 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:59.393861 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b74d4-77df-4e13-9ff8-d38a30543753 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:59.394974 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b74d4-77df-4e13-9ff8-d38a30543753 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:07:59.395349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b74d4-77df-4e13-9ff8-d38a30543753 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:07:59.397473 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b74d4-77df-4e13-9ff8-d38a30543753 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:07:59.418545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 17/34] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:07:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:07:59.420414 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f8682-f6a4-45e5-a30b-7205c12ca154 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:59.420885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f8682-f6a4-45e5-a30b-7205c12ca154 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:59.421240 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f8682-f6a4-45e5-a30b-7205c12ca154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:59.444760 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e8028c-8acf-4021-99d1-f20fc52b85cb None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:07:59.446761 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-dc3f8682-f6a4-45e5-a30b-7205c12ca154 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:07:59.449058 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-dc3f8682-f6a4-45e5-a30b-7205c12ca154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:07:59.483530 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf82510-6617-44a9-927a-c6d92a88a36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:07:59.484576 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf82510-6617-44a9-927a-c6d92a88a36e None admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:59.485157 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf82510-6617-44a9-927a-c6d92a88a36e None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:59.485656 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf82510-6617-44a9-927a-c6d92a88a36e None admin] PATH_INFO: `/v3/domains/default/users/32924f9a30fb4e7b835bb29d116319d9/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:59.488537 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf82510-6617-44a9-927a-c6d92a88a36e None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=32924f9a30fb4e7b835bb29d116319d9, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:07:59.490374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf82510-6617-44a9-927a-c6d92a88a36e None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:07:59.507316 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:07:59.542052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 18/35] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:07:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:07:59.542052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f8682-f6a4-45e5-a30b-7205c12ca154 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:07:59.556306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:07:59.654872 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f8682-f6a4-45e5-a30b-7205c12ca154 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:07:59.655422 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 18/36] 10.222.0.46 () {58 vars in 1361 bytes} [Wed Jul 27 04:07:59 2022] PUT /identity/v3/domains/default/users/32924f9a30fb4e7b835bb29d116319d9/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:07:59.656794 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf82510-6617-44a9-927a-c6d92a88a36e None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:07:59.704769 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c838262-2530-407d-bcd4-bdb4c8c6fd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:07:59.705749 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c838262-2530-407d-bcd4-bdb4c8c6fd0c None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:59.706204 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c838262-2530-407d-bcd4-bdb4c8c6fd0c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:59.706662 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c838262-2530-407d-bcd4-bdb4c8c6fd0c None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:59.708422 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c838262-2530-407d-bcd4-bdb4c8c6fd0c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:07:59.710695 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c838262-2530-407d-bcd4-bdb4c8c6fd0c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:07:59.789602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf82510-6617-44a9-927a-c6d92a88a36e None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:07:59.829326 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d9a8a-2cd2-4f03-825f-9231bc4a73d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:07:59.830379 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d9a8a-2cd2-4f03-825f-9231bc4a73d9 None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:59.831387 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d9a8a-2cd2-4f03-825f-9231bc4a73d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:59.831813 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d9a8a-2cd2-4f03-825f-9231bc4a73d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:59.832720 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d9a8a-2cd2-4f03-825f-9231bc4a73d9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:07:59.834744 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d9a8a-2cd2-4f03-825f-9231bc4a73d9 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:07:59.845322 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 19/37] 10.222.0.46 () {60 vars in 1131 bytes} [Wed Jul 27 04:07:59 2022] POST /identity/v3/roles => generated 219 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:07:59.847543 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c838262-2530-407d-bcd4-bdb4c8c6fd0c None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:07:59.884514 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 19/38] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Jul 27 04:07:59 2022] POST /identity/v3/projects => generated 301 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:07:59.894375 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0443b-4c73-4d35-9635-435821727e96 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:59.895080 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0443b-4c73-4d35-9635-435821727e96 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:59.895476 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0443b-4c73-4d35-9635-435821727e96 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:59.897219 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 20/39] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:07:59 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:07:59.917131 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae1c30-05b6-4249-a0e1-186f1cb5f6ac None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:07:59.917482 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae1c30-05b6-4249-a0e1-186f1cb5f6ac None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:07:59.917705 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae1c30-05b6-4249-a0e1-186f1cb5f6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:07:59.963317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-bbae1c30-05b6-4249-a0e1-186f1cb5f6ac None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:07:59.963511 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:07:59.964851 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-bbae1c30-05b6-4249-a0e1-186f1cb5f6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:00.008729 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c838262-2530-407d-bcd4-bdb4c8c6fd0c None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:00.047496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:00.064293 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b475d428-4217-4764-8067-a3471a96bf80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:00.066244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475d428-4217-4764-8067-a3471a96bf80 None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:00.066532 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475d428-4217-4764-8067-a3471a96bf80 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:00.067119 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475d428-4217-4764-8067-a3471a96bf80 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:00.068497 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b475d428-4217-4764-8067-a3471a96bf80 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:00.070782 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b475d428-4217-4764-8067-a3471a96bf80 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:00.082427 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 21/40] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:07:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:00.082955 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802efacc-801d-475e-87dd-4ef4a0cf1d52 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:00.083226 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802efacc-801d-475e-87dd-4ef4a0cf1d52 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:00.083528 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802efacc-801d-475e-87dd-4ef4a0cf1d52 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:00.085180 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 22/41] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:00.092075 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7c95d-e521-404f-ac1d-f8a746e0993c None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:00.092585 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7c95d-e521-404f-ac1d-f8a746e0993c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:00.092987 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7c95d-e521-404f-ac1d-f8a746e0993c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:00.095559 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 20/42] 10.222.0.46 () {60 vars in 1131 bytes} [Wed Jul 27 04:07:59 2022] POST /identity/v3/roles => generated 225 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:08:00.117754 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b475d428-4217-4764-8067-a3471a96bf80 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:00.129118 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-f6e7c95d-e521-404f-ac1d-f8a746e0993c None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:00.131087 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f6e7c95d-e521-404f-ac1d-f8a746e0993c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:00.236497 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 23/43] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:00.263223 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7c95d-e521-404f-ac1d-f8a746e0993c None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:00.273514 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b475d428-4217-4764-8067-a3471a96bf80 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:00.313132 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:00.323520 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6ea21-767a-4b68-8fd1-8a10a861abab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:00.324261 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6ea21-767a-4b68-8fd1-8a10a861abab None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:00.324661 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6ea21-767a-4b68-8fd1-8a10a861abab None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:00.325005 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6ea21-767a-4b68-8fd1-8a10a861abab None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:00.325851 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6ea21-767a-4b68-8fd1-8a10a861abab None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:00.327923 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6ea21-767a-4b68-8fd1-8a10a861abab None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:00.352193 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 21/44] 10.222.0.46 () {60 vars in 1131 bytes} [Wed Jul 27 04:08:00 2022] POST /identity/v3/roles => generated 223 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:08:00.406227 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7c95d-e521-404f-ac1d-f8a746e0993c None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:00.420026 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:00.449559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3660bc97-e740-4b8b-a693-796c0d2f414c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:00.450266 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660bc97-e740-4b8b-a693-796c0d2f414c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:00.450560 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660bc97-e740-4b8b-a693-796c0d2f414c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:00.453374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660bc97-e740-4b8b-a693-796c0d2f414c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:00.454104 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3660bc97-e740-4b8b-a693-796c0d2f414c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:00.455707 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3660bc97-e740-4b8b-a693-796c0d2f414c None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:00.458606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 24/45] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:08:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:08:00.464716 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6ea21-767a-4b68-8fd1-8a10a861abab None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:00.574496 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6ea21-767a-4b68-8fd1-8a10a861abab None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:00.609571 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198437e5-8a7a-49c0-953d-4d3a5f14d922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:00.610239 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198437e5-8a7a-49c0-953d-4d3a5f14d922 None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:00.610524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198437e5-8a7a-49c0-953d-4d3a5f14d922 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:00.610797 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198437e5-8a7a-49c0-953d-4d3a5f14d922 None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:00.611485 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198437e5-8a7a-49c0-953d-4d3a5f14d922 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:00.613607 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198437e5-8a7a-49c0-953d-4d3a5f14d922 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:00.642354 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 22/46] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Jul 27 04:08:00 2022] POST /identity/v3/projects => generated 312 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:08:01.995969 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227345c6-3bde-41f0-9db9-4234e6906b0f None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:01.996502 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227345c6-3bde-41f0-9db9-4234e6906b0f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:01.996502 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227345c6-3bde-41f0-9db9-4234e6906b0f None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:01.997871 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 25/47] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:02.003412 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93764146-23c5-4592-9737-7ab81df643dd None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:02.003780 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93764146-23c5-4592-9737-7ab81df643dd None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:02.004044 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93764146-23c5-4592-9737-7ab81df643dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:02.022803 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-93764146-23c5-4592-9737-7ab81df643dd None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:02.024318 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-93764146-23c5-4592-9737-7ab81df643dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:02.063818 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:02.091927 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 23/48] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:02.115419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227345c6-3bde-41f0-9db9-4234e6906b0f None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:02.215666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227345c6-3bde-41f0-9db9-4234e6906b0f None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:02.249119 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6945ce-687c-4ca0-aa5e-dbc33d231829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:02.249693 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6945ce-687c-4ca0-aa5e-dbc33d231829 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:02.249992 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6945ce-687c-4ca0-aa5e-dbc33d231829 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:02.250285 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6945ce-687c-4ca0-aa5e-dbc33d231829 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:02.260356 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6945ce-687c-4ca0-aa5e-dbc33d231829 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:02.265493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-4e6945ce-687c-4ca0-aa5e-dbc33d231829 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:08:02.265493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:02.265493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:02.265493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:02.265493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:02.265493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:02.265493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:02.265493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:02.265493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:02.265493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:02.265493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:02.265493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:02.265493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:02.265493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:02.265493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:02.265493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:02.266360 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:02.266360 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:02.266360 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:02.266360 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:02.266360 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:02.266360 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:02.266360 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:02.266360 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:02.266360 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:02.266360 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:02.266360 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:02.266360 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:02.266360 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:02.266360 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:02.266360 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:02.266360 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:02.267181 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:02.267181 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:02.267181 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:02.267181 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:02.267181 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:02.267181 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:02.267181 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:02.267181 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:02.267181 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:02.267181 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:02.267181 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:08:02.267181 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:08:02.267181 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 26/49] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:08:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:02.273042 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93764146-23c5-4592-9737-7ab81df643dd None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:02.369594 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93764146-23c5-4592-9737-7ab81df643dd None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:02.402930 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eac689-8632-40c6-bbaf-dfe913530d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:02.403561 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eac689-8632-40c6-bbaf-dfe913530d46 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:02.403852 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eac689-8632-40c6-bbaf-dfe913530d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:02.404136 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eac689-8632-40c6-bbaf-dfe913530d46 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:02.404867 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eac689-8632-40c6-bbaf-dfe913530d46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:02.405180 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eac689-8632-40c6-bbaf-dfe913530d46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:02.406606 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eac689-8632-40c6-bbaf-dfe913530d46 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:02.415426 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 24/50] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:02.420908 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6945ce-687c-4ca0-aa5e-dbc33d231829 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:02.462972 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:02.523139 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6945ce-687c-4ca0-aa5e-dbc33d231829 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:02.556870 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f9408-ed0e-469e-8735-c4519697021f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:02.557522 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f9408-ed0e-469e-8735-c4519697021f None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:02.557808 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f9408-ed0e-469e-8735-c4519697021f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:02.558120 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f9408-ed0e-469e-8735-c4519697021f None admin] PATH_INFO: `/v3/users/32924f9a30fb4e7b835bb29d116319d9` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:02.559288 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f9408-ed0e-469e-8735-c4519697021f None admin] RBAC: Authorizing `identity:get_user(user_id=32924f9a30fb4e7b835bb29d116319d9)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:02.560749 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f9408-ed0e-469e-8735-c4519697021f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:02.562380 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 27/51] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:02 2022] GET /identity/v3/users/32924f9a30fb4e7b835bb29d116319d9 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:02.568653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eac689-8632-40c6-bbaf-dfe913530d46 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:02.605825 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:02.652494 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002570f-ca22-4d5b-b1df-472b8c34fc50 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:02.652906 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002570f-ca22-4d5b-b1df-472b8c34fc50 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:02.653192 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002570f-ca22-4d5b-b1df-472b8c34fc50 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:02.654593 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 28/52] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 04:08:02.659476 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f4864-d46a-417e-8e68-b2179ae26c0f None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:02.659769 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f4864-d46a-417e-8e68-b2179ae26c0f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:02.660040 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f4864-d46a-417e-8e68-b2179ae26c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:02.672298 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eac689-8632-40c6-bbaf-dfe913530d46 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:02.677735 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-898f4864-d46a-417e-8e68-b2179ae26c0f None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:02.678953 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-898f4864-d46a-417e-8e68-b2179ae26c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:02.706659 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46edb5ec-be55-49d6-aec6-9bf2c435a310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:02.707305 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46edb5ec-be55-49d6-aec6-9bf2c435a310 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:02.707587 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46edb5ec-be55-49d6-aec6-9bf2c435a310 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:02.707864 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46edb5ec-be55-49d6-aec6-9bf2c435a310 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:02.709012 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46edb5ec-be55-49d6-aec6-9bf2c435a310 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:02.710507 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46edb5ec-be55-49d6-aec6-9bf2c435a310 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:02.712083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 25/53] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:08:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:02.717783 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46edb5ec-be55-49d6-aec6-9bf2c435a310 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:02.747931 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 29/54] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 04:08:02.772952 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f4864-d46a-417e-8e68-b2179ae26c0f None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:02.805665 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:02.815820 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46edb5ec-be55-49d6-aec6-9bf2c435a310 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:02.818174 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:02.849371 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947de1a-aa4b-497b-a2c3-bb16aab7942c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:02.850014 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947de1a-aa4b-497b-a2c3-bb16aab7942c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:02.850300 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947de1a-aa4b-497b-a2c3-bb16aab7942c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:02.850580 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947de1a-aa4b-497b-a2c3-bb16aab7942c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:02.851370 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e947de1a-aa4b-497b-a2c3-bb16aab7942c None admin] RBAC: Adding query filter params (role.id=cf771a8e94ee46149eb081347fea6d67, scope.domain.id=default, user.id=32924f9a30fb4e7b835bb29d116319d9) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:02.851679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e947de1a-aa4b-497b-a2c3-bb16aab7942c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:02.853121 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e947de1a-aa4b-497b-a2c3-bb16aab7942c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:02.860187 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 26/55] 10.222.0.46 () {56 vars in 1322 bytes} [Wed Jul 27 04:08:02 2022] GET /identity/v3/role_assignments?user.id=32924f9a30fb4e7b835bb29d116319d9&scope.domain.id=default&role.id=cf771a8e94ee46149eb081347fea6d67 => generated 530 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:08:02.871881 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f4864-d46a-417e-8e68-b2179ae26c0f None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:02.908671 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326f18f-cb78-4628-aba8-c1d4882babfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:02.909379 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326f18f-cb78-4628-aba8-c1d4882babfd None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:02.909780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326f18f-cb78-4628-aba8-c1d4882babfd None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:02.910103 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326f18f-cb78-4628-aba8-c1d4882babfd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:02.911384 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e326f18f-cb78-4628-aba8-c1d4882babfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:02.912895 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e326f18f-cb78-4628-aba8-c1d4882babfd None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:02.914664 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 30/56] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:08:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:08:02.920515 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947de1a-aa4b-497b-a2c3-bb16aab7942c None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:03.026220 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947de1a-aa4b-497b-a2c3-bb16aab7942c None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:03.060179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc5294-54ff-4e72-bdea-6de5fef55932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:03.060743 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc5294-54ff-4e72-bdea-6de5fef55932 None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:03.061031 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc5294-54ff-4e72-bdea-6de5fef55932 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:03.061313 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc5294-54ff-4e72-bdea-6de5fef55932 None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:03.062034 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc5294-54ff-4e72-bdea-6de5fef55932 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:03.063758 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc5294-54ff-4e72-bdea-6de5fef55932 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:03.090050 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 27/57] 10.222.0.46 () {60 vars in 1140 bytes} [Wed Jul 27 04:08:02 2022] POST /identity/v3/projects => generated 298 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:08:05.064845 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b1fa6-573c-40f1-baac-2204c2b03cb2 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:05.065358 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b1fa6-573c-40f1-baac-2204c2b03cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:05.065499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b1fa6-573c-40f1-baac-2204c2b03cb2 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:05.066963 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 31/58] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:05.071939 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13316567-d51b-4686-99ed-fbccbde78f69 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:05.072336 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13316567-d51b-4686-99ed-fbccbde78f69 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:05.072501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13316567-d51b-4686-99ed-fbccbde78f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:05.090456 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-13316567-d51b-4686-99ed-fbccbde78f69 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:05.091634 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-13316567-d51b-4686-99ed-fbccbde78f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:05.105890 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:05.163283 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 28/59] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:05.188648 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b1fa6-573c-40f1-baac-2204c2b03cb2 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:05.288047 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b1fa6-573c-40f1-baac-2204c2b03cb2 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:05.290348 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:05.321764 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81cdf9e-f788-463d-a111-4a81f3e9b50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:05.322374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81cdf9e-f788-463d-a111-4a81f3e9b50d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:05.322661 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81cdf9e-f788-463d-a111-4a81f3e9b50d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:05.322940 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81cdf9e-f788-463d-a111-4a81f3e9b50d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:05.324101 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81cdf9e-f788-463d-a111-4a81f3e9b50d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:05.325552 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81cdf9e-f788-463d-a111-4a81f3e9b50d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:05.327162 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 32/60] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:08:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:05.333241 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13316567-d51b-4686-99ed-fbccbde78f69 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:05.428331 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13316567-d51b-4686-99ed-fbccbde78f69 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:05.461636 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d922271-4d47-487e-858a-a2a7f724e1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:05.462302 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d922271-4d47-487e-858a-a2a7f724e1e9 None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:05.462597 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d922271-4d47-487e-858a-a2a7f724e1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:05.462879 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d922271-4d47-487e-858a-a2a7f724e1e9 None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:05.463590 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d922271-4d47-487e-858a-a2a7f724e1e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:05.465038 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d922271-4d47-487e-858a-a2a7f724e1e9 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:05.491243 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 29/61] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 27 04:08:05 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:08:08.538774 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab569a-7d62-4fd0-8a25-c2794178dedc None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:08.539646 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab569a-7d62-4fd0-8a25-c2794178dedc None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:08.540071 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab569a-7d62-4fd0-8a25-c2794178dedc None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:08.542077 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed382c8-ec34-4a26-88a6-ab9a5a79cadf None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:08.542475 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 33/62] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:08.543019 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed382c8-ec34-4a26-88a6-ab9a5a79cadf None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:08.543520 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed382c8-ec34-4a26-88a6-ab9a5a79cadf None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:08.545108 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 30/63] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:08.546698 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fa245-7037-4a98-8bdd-e59e865fc5be None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:08.546989 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fa245-7037-4a98-8bdd-e59e865fc5be None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:08.547267 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fa245-7037-4a98-8bdd-e59e865fc5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:08.550952 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c402d6-981e-4713-b231-9e1a4d51e6ac None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:08.551480 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c402d6-981e-4713-b231-9e1a4d51e6ac None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:08.551962 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c402d6-981e-4713-b231-9e1a4d51e6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:08.565643 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:08.572570 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-d7c402d6-981e-4713-b231-9e1a4d51e6ac None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:08.572725 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-b79fa245-7037-4a98-8bdd-e59e865fc5be None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:08.573774 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b79fa245-7037-4a98-8bdd-e59e865fc5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:08.574408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d7c402d6-981e-4713-b231-9e1a4d51e6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:08.682081 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 34/64] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:08.705163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79fa245-7037-4a98-8bdd-e59e865fc5be None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:08.720686 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 31/65] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 04:08:08.754810 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c402d6-981e-4713-b231-9e1a4d51e6ac None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:08.866960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c402d6-981e-4713-b231-9e1a4d51e6ac None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:08.877708 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79fa245-7037-4a98-8bdd-e59e865fc5be None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:08.881137 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:08.934502 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79500ee9-e719-4277-bddd-5268062c0cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:08.935458 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79500ee9-e719-4277-bddd-5268062c0cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:08.936528 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79500ee9-e719-4277-bddd-5268062c0cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:08.937159 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79500ee9-e719-4277-bddd-5268062c0cf6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:08.942529 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:08.948066 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d9638-4d60-4425-b798-bacb10eb808b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:08.948778 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d9638-4d60-4425-b798-bacb10eb808b None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:08.949063 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d9638-4d60-4425-b798-bacb10eb808b None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:08.949384 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d9638-4d60-4425-b798-bacb10eb808b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:08.963933 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79500ee9-e719-4277-bddd-5268062c0cf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:08.971083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-79500ee9-e719-4277-bddd-5268062c0cf6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:08.971083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:08.971083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:08.971083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:08.971083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:08.971083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:08.971083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:08.971083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:08.971083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:08.971083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:08.971083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:08.971083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:08.971083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:08.971083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:08.971083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:08.971083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:08.972738 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:08.972738 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:08.972738 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:08.972738 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:08.972738 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:08.972738 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:08.972738 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:08.972738 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:08.972738 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:08.972738 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:08.972738 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:08.972738 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:08.972738 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:08.972738 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:08.972738 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:08.972738 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:08.973700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:08.973700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:08.973700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:08.973700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:08.973700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:08.973700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:08.973700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:08.973700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:08.973700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:08.973700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:08.973700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:08.973700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:08:08.973700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 32/66] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:08:08.981878 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762d9638-4d60-4425-b798-bacb10eb808b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:08.983752 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79500ee9-e719-4277-bddd-5268062c0cf6 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:08.987606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-762d9638-4d60-4425-b798-bacb10eb808b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:08:08.987606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:08.987606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:08.987606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:08.987606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:08.987606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:08.987606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:08.987606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:08.987606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:08.987606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:08.987606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:08.987606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:08.987606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:08.987606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:08.987606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:08.987606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:08.988637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:08.988637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:08.988637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:08.988637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:08.988637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:08.988637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:08.988637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:08.988637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:08.988637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:08.988637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:08.988637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:08.988637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:08.988637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:08.988637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:08.988637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:08.988637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:08.989499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:08.989499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:08.989499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:08.989499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:08.989499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:08.989499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:08.989499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:08.989499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:08.989499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:08.989499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:08.989499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:08:08.989499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:08:08.991342 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 35/67] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:08:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:08:08.999909 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d9638-4d60-4425-b798-bacb10eb808b None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:09.115845 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79500ee9-e719-4277-bddd-5268062c0cf6 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:09.125311 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d9638-4d60-4425-b798-bacb10eb808b None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:09.163279 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab7fee-4c27-40bf-b548-dd7817feffe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:09.164279 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab7fee-4c27-40bf-b548-dd7817feffe3 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:09.164780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab7fee-4c27-40bf-b548-dd7817feffe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:09.165247 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab7fee-4c27-40bf-b548-dd7817feffe3 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:09.165760 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b4be4-6746-4dbc-b124-32c51762da31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:09.166424 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b4be4-6746-4dbc-b124-32c51762da31 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:09.166626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ab7fee-4c27-40bf-b548-dd7817feffe3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:09.166735 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b4be4-6746-4dbc-b124-32c51762da31 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:09.166983 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b4be4-6746-4dbc-b124-32c51762da31 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:09.167417 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ab7fee-4c27-40bf-b548-dd7817feffe3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:09.167756 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b4be4-6746-4dbc-b124-32c51762da31 None admin] RBAC: Adding query filter params (name=member) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:09.168019 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b4be4-6746-4dbc-b124-32c51762da31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:09.169412 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b4be4-6746-4dbc-b124-32c51762da31 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:09.169667 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ab7fee-4c27-40bf-b548-dd7817feffe3 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:09.179888 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 36/68] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:09.180036 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 33/69] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 27 04:08:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:08:09.186028 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac24415-3072-4d57-9e74-b82745fad3e9 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:09.186194 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab7fee-4c27-40bf-b548-dd7817feffe3 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:09.186274 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac24415-3072-4d57-9e74-b82745fad3e9 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:09.186274 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac24415-3072-4d57-9e74-b82745fad3e9 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:09.187239 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 34/70] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:09.189119 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac24415-3072-4d57-9e74-b82745fad3e9 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:09.218639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:09.254358 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:09.300570 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac24415-3072-4d57-9e74-b82745fad3e9 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:09.314115 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab7fee-4c27-40bf-b548-dd7817feffe3 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:09.333114 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d970ec6-5f70-4484-86eb-68fa4e054182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:09.333765 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d970ec6-5f70-4484-86eb-68fa4e054182 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:09.334083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d970ec6-5f70-4484-86eb-68fa4e054182 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:09.334363 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d970ec6-5f70-4484-86eb-68fa4e054182 None admin] PATH_INFO: `/v3/users/706e785a4bca46e7ab7003c417e51cc1` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:09.347505 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9843bcd-1ccb-484a-ac02-17fe3595ddad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:09.347720 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9843bcd-1ccb-484a-ac02-17fe3595ddad None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:09.347999 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9843bcd-1ccb-484a-ac02-17fe3595ddad None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:09.348118 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d970ec6-5f70-4484-86eb-68fa4e054182 None admin] RBAC: Authorizing `identity:get_user(user_id=706e785a4bca46e7ab7003c417e51cc1)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:09.348271 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9843bcd-1ccb-484a-ac02-17fe3595ddad None admin] PATH_INFO: `/v3/users/32924f9a30fb4e7b835bb29d116319d9` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:09.349448 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9843bcd-1ccb-484a-ac02-17fe3595ddad None admin] RBAC: Authorizing `identity:get_user(user_id=32924f9a30fb4e7b835bb29d116319d9)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:09.349542 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d970ec6-5f70-4484-86eb-68fa4e054182 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:09.350906 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9843bcd-1ccb-484a-ac02-17fe3595ddad None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:09.351198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 35/71] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:09 2022] GET /identity/v3/users/706e785a4bca46e7ab7003c417e51cc1 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:08:09.356674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 37/72] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:09 2022] GET /identity/v3/users/32924f9a30fb4e7b835bb29d116319d9 => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:09.356674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cba84e-cb92-483e-a971-d2c41533c751 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:09.356674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cba84e-cb92-483e-a971-d2c41533c751 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:09.356674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cba84e-cb92-483e-a971-d2c41533c751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:09.357202 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96181261-d27d-4bce-8a2e-461758ccd1f0 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:09.357471 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96181261-d27d-4bce-8a2e-461758ccd1f0 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:09.357753 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96181261-d27d-4bce-8a2e-461758ccd1f0 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:09.359033 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 36/73] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:09.360123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c7015-94aa-4154-9fff-5a00fe635ebe None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:09.360494 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c7015-94aa-4154-9fff-5a00fe635ebe None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:09.360765 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c7015-94aa-4154-9fff-5a00fe635ebe None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:09.362605 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 37/74] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:09.363269 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c7015-94aa-4154-9fff-5a00fe635ebe None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:09.377458 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-b3cba84e-cb92-483e-a971-d2c41533c751 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:09.378920 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b3cba84e-cb92-483e-a971-d2c41533c751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:09.444860 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 38/75] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:09.445703 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cba84e-cb92-483e-a971-d2c41533c751 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:09.472708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c7015-94aa-4154-9fff-5a00fe635ebe None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:09.479563 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:09.489843 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:09.507046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3a61d8-7416-44af-b7be-694a4734ad6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:09.507696 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a61d8-7416-44af-b7be-694a4734ad6b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:09.507979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a61d8-7416-44af-b7be-694a4734ad6b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:09.508254 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a61d8-7416-44af-b7be-694a4734ad6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:09.554486 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cba84e-cb92-483e-a971-d2c41533c751 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:09.575461 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3a61d8-7416-44af-b7be-694a4734ad6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:09.576946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3a61d8-7416-44af-b7be-694a4734ad6b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:09.587834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef3e64a-59d6-439b-9f11-747054c2190b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:09.588457 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3e64a-59d6-439b-9f11-747054c2190b None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:09.588736 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3e64a-59d6-439b-9f11-747054c2190b None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:09.589015 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3e64a-59d6-439b-9f11-747054c2190b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:09.656129 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef3e64a-59d6-439b-9f11-747054c2190b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:09.657598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef3e64a-59d6-439b-9f11-747054c2190b None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:09.673500 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 38/76] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:08:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:08:09.674959 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15cf4ab-e275-44bf-b3fe-749ffdf594cb None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:09.675251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15cf4ab-e275-44bf-b3fe-749ffdf594cb None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:09.675527 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15cf4ab-e275-44bf-b3fe-749ffdf594cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:09.682450 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:09.692684 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-d15cf4ab-e275-44bf-b3fe-749ffdf594cb None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:09.693882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d15cf4ab-e275-44bf-b3fe-749ffdf594cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:09.704209 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:09.760154 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 39/77] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:09.761078 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 39/78] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:08:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:08:09.761572 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fc5bc-ab05-46a4-99e2-262b749d15d5 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:09.761865 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fc5bc-ab05-46a4-99e2-262b749d15d5 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:09.762172 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fc5bc-ab05-46a4-99e2-262b749d15d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:09.765866 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef3e64a-59d6-439b-9f11-747054c2190b None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:09.779807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-520fc5bc-ab05-46a4-99e2-262b749d15d5 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:09.780978 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-520fc5bc-ab05-46a4-99e2-262b749d15d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:09.849520 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 40/79] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:09.850413 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fc5bc-ab05-46a4-99e2-262b749d15d5 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:09.881228 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef3e64a-59d6-439b-9f11-747054c2190b None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:09.904062 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:09.917051 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a922a7-7173-40d2-add0-8b7f23fddfa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:09.917709 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a922a7-7173-40d2-add0-8b7f23fddfa4 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:09.918001 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a922a7-7173-40d2-add0-8b7f23fddfa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:09.918286 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a922a7-7173-40d2-add0-8b7f23fddfa4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:09.925492 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:09.928943 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a922a7-7173-40d2-add0-8b7f23fddfa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:09.933843 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-a9a922a7-7173-40d2-add0-8b7f23fddfa4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 04:08:09.933843 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:09.933843 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:09.933843 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:09.933843 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:09.933843 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:09.933843 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:09.933843 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:09.933843 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:09.933843 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:09.933843 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:09.933843 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:09.933843 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:09.933843 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:09.933843 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:09.933843 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:09.934717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:09.934717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:09.934717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:09.934717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:09.934717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:09.934717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:09.934717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:09.934717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:09.934717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:09.934717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:09.934717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:09.934717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:09.934717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:09.934717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:09.934717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:09.934717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:09.935456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:09.935456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:09.935456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:09.935456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:09.935456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:09.935456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:09.935456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:09.935456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:09.935456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:09.935456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:09.935456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 04:08:09.935456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:08:09.935456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 40/80] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 27 04:08:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:09.936153 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a922a7-7173-40d2-add0-8b7f23fddfa4 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:09.948917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fc5bc-ab05-46a4-99e2-262b749d15d5 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:09.982828 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421bd3a-d219-435b-9dd1-0b3ba46fb0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:09.983545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421bd3a-d219-435b-9dd1-0b3ba46fb0d5 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:09.983859 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421bd3a-d219-435b-9dd1-0b3ba46fb0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:09.984143 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421bd3a-d219-435b-9dd1-0b3ba46fb0d5 None admin] PATH_INFO: `/v3/projects/85168a036bf94350836ae48489878ed3` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:09.985311 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5421bd3a-d219-435b-9dd1-0b3ba46fb0d5 None admin] RBAC: Authorizing `identity:get_project(project_id=85168a036bf94350836ae48489878ed3)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:09.986815 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5421bd3a-d219-435b-9dd1-0b3ba46fb0d5 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:09.989097 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 41/81] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:09 2022] GET /identity/v3/projects/85168a036bf94350836ae48489878ed3 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:09.989868 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421bd3a-d219-435b-9dd1-0b3ba46fb0d5 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:10.040630 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a922a7-7173-40d2-add0-8b7f23fddfa4 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:10.076986 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c71b53-17a1-4855-a0a2-d0c8912c7c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:10.077770 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c71b53-17a1-4855-a0a2-d0c8912c7c45 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:10.078162 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c71b53-17a1-4855-a0a2-d0c8912c7c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:10.078520 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c71b53-17a1-4855-a0a2-d0c8912c7c45 None admin] PATH_INFO: `/v3/projects/85168a036bf94350836ae48489878ed3` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:10.079788 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c71b53-17a1-4855-a0a2-d0c8912c7c45 None admin] RBAC: Authorizing `identity:get_project(project_id=85168a036bf94350836ae48489878ed3)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:10.081956 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c71b53-17a1-4855-a0a2-d0c8912c7c45 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:10.083757 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 41/82] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:09 2022] GET /identity/v3/projects/85168a036bf94350836ae48489878ed3 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:10.084606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c71b53-17a1-4855-a0a2-d0c8912c7c45 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:10.103014 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421bd3a-d219-435b-9dd1-0b3ba46fb0d5 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:10.142477 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:10.146157 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823547b9-8935-4b7b-8161-39d2cbe6293f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:10.147020 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823547b9-8935-4b7b-8161-39d2cbe6293f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:10.147450 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823547b9-8935-4b7b-8161-39d2cbe6293f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:10.147871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823547b9-8935-4b7b-8161-39d2cbe6293f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:10.158930 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823547b9-8935-4b7b-8161-39d2cbe6293f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:10.164359 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-823547b9-8935-4b7b-8161-39d2cbe6293f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:10.164359 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:10.164359 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:10.164359 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:10.164359 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:10.164359 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:10.164359 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:10.164359 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:10.164359 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:10.164359 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:10.164359 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:10.164359 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:10.164359 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:10.164359 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:10.164359 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:10.164359 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:10.165407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:10.165407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:10.165407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:10.165407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:10.165407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:10.165407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:10.165407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:10.165407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:10.165407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:10.165407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:10.165407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:10.165407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:10.165407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:10.165407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:10.165407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:10.165407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:10.166416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:10.166416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:10.166416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:10.166416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:10.166416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:10.166416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:10.166416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:10.166416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:10.166416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:10.166416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:10.166416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:10.166416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:08:10.166416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 42/83] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:10.166416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823547b9-8935-4b7b-8161-39d2cbe6293f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:10.171112 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:10.201876 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c71b53-17a1-4855-a0a2-d0c8912c7c45 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:10.236638 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1762d16-b72e-4353-b9d9-8294d21e6cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:10.237907 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1762d16-b72e-4353-b9d9-8294d21e6cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:10.238227 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1762d16-b72e-4353-b9d9-8294d21e6cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:10.238504 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1762d16-b72e-4353-b9d9-8294d21e6cc3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:10.239670 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1762d16-b72e-4353-b9d9-8294d21e6cc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:10.241118 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1762d16-b72e-4353-b9d9-8294d21e6cc3 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:10.242778 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 42/84] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:08:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:10.243563 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1762d16-b72e-4353-b9d9-8294d21e6cc3 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:10.275017 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823547b9-8935-4b7b-8161-39d2cbe6293f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:10.313315 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f06b97-6ef2-4a83-b5d7-ecafb80c883b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:10.314005 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f06b97-6ef2-4a83-b5d7-ecafb80c883b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:10.314305 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f06b97-6ef2-4a83-b5d7-ecafb80c883b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:10.314590 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f06b97-6ef2-4a83-b5d7-ecafb80c883b None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:10.315517 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f06b97-6ef2-4a83-b5d7-ecafb80c883b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:10.315836 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f06b97-6ef2-4a83-b5d7-ecafb80c883b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:10.317230 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f06b97-6ef2-4a83-b5d7-ecafb80c883b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:10.325750 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 43/85] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:08:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:10.326599 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f06b97-6ef2-4a83-b5d7-ecafb80c883b None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:10.351643 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1762d16-b72e-4353-b9d9-8294d21e6cc3 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:10.368902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:10.385889 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ffe6e-81ee-457d-abf4-4d4c490e4792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:10.386551 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ffe6e-81ee-457d-abf4-4d4c490e4792 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:10.386834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ffe6e-81ee-457d-abf4-4d4c490e4792 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:10.387107 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ffe6e-81ee-457d-abf4-4d4c490e4792 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:10.387897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ffe6e-81ee-457d-abf4-4d4c490e4792 None admin] RBAC: Adding query filter params (role.id=7b6f2ade82434ee79175385de17049eb, scope.project.id=85168a036bf94350836ae48489878ed3, user.id=706e785a4bca46e7ab7003c417e51cc1) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:10.388203 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ffe6e-81ee-457d-abf4-4d4c490e4792 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:10.389639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ffe6e-81ee-457d-abf4-4d4c490e4792 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:10.395658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 43/86] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:08:10 2022] GET /identity/v3/role_assignments?user.id=706e785a4bca46e7ab7003c417e51cc1&scope.project.id=85168a036bf94350836ae48489878ed3&role.id=7b6f2ade82434ee79175385de17049eb => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:10.396441 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ffe6e-81ee-457d-abf4-4d4c490e4792 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:10.412437 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:10.442186 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f06b97-6ef2-4a83-b5d7-ecafb80c883b None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:10.485707 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b07b7-aebd-4a5e-8031-478697fbd24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:10.486427 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b07b7-aebd-4a5e-8031-478697fbd24c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:10.487500 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b07b7-aebd-4a5e-8031-478697fbd24c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:10.487500 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b07b7-aebd-4a5e-8031-478697fbd24c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:10.487879 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b07b7-aebd-4a5e-8031-478697fbd24c None admin] RBAC: Adding query filter params (role.id=cf771a8e94ee46149eb081347fea6d67, scope.project.id=85168a036bf94350836ae48489878ed3, user.id=32924f9a30fb4e7b835bb29d116319d9) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:10.488201 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b07b7-aebd-4a5e-8031-478697fbd24c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:10.489672 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b07b7-aebd-4a5e-8031-478697fbd24c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:10.496122 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 44/87] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:08:10 2022] GET /identity/v3/role_assignments?user.id=32924f9a30fb4e7b835bb29d116319d9&scope.project.id=85168a036bf94350836ae48489878ed3&role.id=cf771a8e94ee46149eb081347fea6d67 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:10.503270 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b07b7-aebd-4a5e-8031-478697fbd24c None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:10.540763 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ffe6e-81ee-457d-abf4-4d4c490e4792 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:10.595933 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d5a52-5010-4570-969d-251a882b844f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:10.596746 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d5a52-5010-4570-969d-251a882b844f None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:10.597103 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d5a52-5010-4570-969d-251a882b844f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:10.597202 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d5a52-5010-4570-969d-251a882b844f None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:10.598325 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d5a52-5010-4570-969d-251a882b844f None admin] RBAC: Adding query filter params (name=member) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:10.598639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d5a52-5010-4570-969d-251a882b844f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:10.600387 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d5a52-5010-4570-969d-251a882b844f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:10.613461 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 44/88] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 27 04:08:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:10.614482 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d5a52-5010-4570-969d-251a882b844f None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:10.654112 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b07b7-aebd-4a5e-8031-478697fbd24c None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:10.674381 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:10.685111 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:10.693677 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a4497-93f2-4644-bf34-91fe5ceecb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:10.694343 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a4497-93f2-4644-bf34-91fe5ceecb15 None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:10.694626 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a4497-93f2-4644-bf34-91fe5ceecb15 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:10.694901 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a4497-93f2-4644-bf34-91fe5ceecb15 None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:10.695601 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a4497-93f2-4644-bf34-91fe5ceecb15 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:10.697060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a4497-93f2-4644-bf34-91fe5ceecb15 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:10.730110 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 45/89] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 27 04:08:10 2022] POST /identity/v3/users => generated 289 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:08:10.730110 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a4497-93f2-4644-bf34-91fe5ceecb15 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:10.759745 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d5a52-5010-4570-969d-251a882b844f None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:10.856184 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a85e6-af4d-4bbe-a4ae-a08b04155914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:10.856932 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a85e6-af4d-4bbe-a4ae-a08b04155914 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:10.857220 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a85e6-af4d-4bbe-a4ae-a08b04155914 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:10.857490 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a85e6-af4d-4bbe-a4ae-a08b04155914 None admin] PATH_INFO: `/v3/users/706e785a4bca46e7ab7003c417e51cc1` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:10.860972 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a85e6-af4d-4bbe-a4ae-a08b04155914 None admin] RBAC: Authorizing `identity:get_user(user_id=706e785a4bca46e7ab7003c417e51cc1)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:10.862756 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a85e6-af4d-4bbe-a4ae-a08b04155914 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:10.864350 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 45/90] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:10 2022] GET /identity/v3/users/706e785a4bca46e7ab7003c417e51cc1 => generated 275 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:10.870112 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a85e6-af4d-4bbe-a4ae-a08b04155914 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:10.883590 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a4497-93f2-4644-bf34-91fe5ceecb15 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:10.929462 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da781c-0431-4638-b056-037ce853a504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:10.930497 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da781c-0431-4638-b056-037ce853a504 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:10.930981 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da781c-0431-4638-b056-037ce853a504 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:10.931478 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da781c-0431-4638-b056-037ce853a504 None admin] PATH_INFO: `/v3/users/706e785a4bca46e7ab7003c417e51cc1` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:10.934288 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80da781c-0431-4638-b056-037ce853a504 None admin] RBAC: Authorizing `identity:get_user(user_id=706e785a4bca46e7ab7003c417e51cc1)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:10.936580 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80da781c-0431-4638-b056-037ce853a504 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:10.939885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 46/91] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:10 2022] GET /identity/v3/users/706e785a4bca46e7ab7003c417e51cc1 => generated 275 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:10.945720 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da781c-0431-4638-b056-037ce853a504 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:11.006381 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a85e6-af4d-4bbe-a4ae-a08b04155914 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:11.020209 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:11.022297 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:11.046232 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a713b9-7e99-402c-8410-50ecd113e299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:11.046958 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a713b9-7e99-402c-8410-50ecd113e299 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:11.047312 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a713b9-7e99-402c-8410-50ecd113e299 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:11.047661 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a713b9-7e99-402c-8410-50ecd113e299 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:11.080613 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da781c-0431-4638-b056-037ce853a504 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:11.124631 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc15bc3-b171-40bf-abff-f2ebabf92bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:11.125536 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc15bc3-b171-40bf-abff-f2ebabf92bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:11.126037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc15bc3-b171-40bf-abff-f2ebabf92bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:11.126491 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc15bc3-b171-40bf-abff-f2ebabf92bb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:11.130334 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a713b9-7e99-402c-8410-50ecd113e299 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:11.132143 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a713b9-7e99-402c-8410-50ecd113e299 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:11.212089 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc15bc3-b171-40bf-abff-f2ebabf92bb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:11.214285 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc15bc3-b171-40bf-abff-f2ebabf92bb9 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:11.253204 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 46/92] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:08:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:08:11.258973 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a713b9-7e99-402c-8410-50ecd113e299 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:11.277404 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:11.286786 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:11.340472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 47/93] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:08:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:08:11.346976 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc15bc3-b171-40bf-abff-f2ebabf92bb9 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:11.391654 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a713b9-7e99-402c-8410-50ecd113e299 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:11.440012 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d613260-947c-4dc0-8d44-a5631f231fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:11.441295 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d613260-947c-4dc0-8d44-a5631f231fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:11.441580 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d613260-947c-4dc0-8d44-a5631f231fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:11.441855 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d613260-947c-4dc0-8d44-a5631f231fd8 None admin] PATH_INFO: `/v3/projects/85168a036bf94350836ae48489878ed3` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:11.443116 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d613260-947c-4dc0-8d44-a5631f231fd8 None admin] RBAC: Authorizing `identity:get_project(project_id=85168a036bf94350836ae48489878ed3)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:11.444640 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d613260-947c-4dc0-8d44-a5631f231fd8 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:11.447001 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 47/94] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:11 2022] GET /identity/v3/projects/85168a036bf94350836ae48489878ed3 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:11.455631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d613260-947c-4dc0-8d44-a5631f231fd8 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:11.469457 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc15bc3-b171-40bf-abff-f2ebabf92bb9 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:11.516336 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b09d7-08bf-4a39-ab41-b255c4866262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:11.516995 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b09d7-08bf-4a39-ab41-b255c4866262 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:11.517277 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b09d7-08bf-4a39-ab41-b255c4866262 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:11.517554 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b09d7-08bf-4a39-ab41-b255c4866262 None admin] PATH_INFO: `/v3/projects/e6014d57b86c4cf1848ccb82dc88cea9` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:11.519001 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6b09d7-08bf-4a39-ab41-b255c4866262 None admin] RBAC: Authorizing `identity:get_project(project_id=e6014d57b86c4cf1848ccb82dc88cea9)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:11.520603 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6b09d7-08bf-4a39-ab41-b255c4866262 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:11.522612 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 48/95] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:11 2022] GET /identity/v3/projects/e6014d57b86c4cf1848ccb82dc88cea9 => generated 312 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:11.531023 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b09d7-08bf-4a39-ab41-b255c4866262 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:11.576521 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:11.583957 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:11.618008 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d613260-947c-4dc0-8d44-a5631f231fd8 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:11.657289 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e859314-0dae-4023-8ba9-f29b836b8ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:11.657995 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e859314-0dae-4023-8ba9-f29b836b8ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:11.658282 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e859314-0dae-4023-8ba9-f29b836b8ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:11.658558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e859314-0dae-4023-8ba9-f29b836b8ce4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:11.659926 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e859314-0dae-4023-8ba9-f29b836b8ce4 None admin] RBAC: Adding query filter params (role.id=cdfc87279ea24b65bae3961035133725, scope.project.id=85168a036bf94350836ae48489878ed3, user.id=706e785a4bca46e7ab7003c417e51cc1) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:11.660226 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e859314-0dae-4023-8ba9-f29b836b8ce4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:11.661937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e859314-0dae-4023-8ba9-f29b836b8ce4 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:11.668659 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 48/96] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:08:11 2022] GET /identity/v3/role_assignments?user.id=706e785a4bca46e7ab7003c417e51cc1&scope.project.id=85168a036bf94350836ae48489878ed3&role.id=cdfc87279ea24b65bae3961035133725 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:08:11.764146 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b09d7-08bf-4a39-ab41-b255c4866262 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:11.815534 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce0357e-5e22-4004-a1f9-08ca2b61afaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:11.816175 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0357e-5e22-4004-a1f9-08ca2b61afaa None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:11.816453 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0357e-5e22-4004-a1f9-08ca2b61afaa None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:11.816728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0357e-5e22-4004-a1f9-08ca2b61afaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:11.817544 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce0357e-5e22-4004-a1f9-08ca2b61afaa None admin] RBAC: Adding query filter params (role.id=7b6f2ade82434ee79175385de17049eb, scope.project.id=e6014d57b86c4cf1848ccb82dc88cea9, user.id=706e785a4bca46e7ab7003c417e51cc1) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:11.817860 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce0357e-5e22-4004-a1f9-08ca2b61afaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:11.819453 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce0357e-5e22-4004-a1f9-08ca2b61afaa None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:11.826073 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 49/97] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:08:11 2022] GET /identity/v3/role_assignments?user.id=706e785a4bca46e7ab7003c417e51cc1&scope.project.id=e6014d57b86c4cf1848ccb82dc88cea9&role.id=7b6f2ade82434ee79175385de17049eb => generated 259 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:12.957476 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7264e03-6b60-4cd9-8ddf-63c6fb4606a5 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:12.958009 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7264e03-6b60-4cd9-8ddf-63c6fb4606a5 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:12.958088 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7264e03-6b60-4cd9-8ddf-63c6fb4606a5 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:12.959407 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 49/98] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 04:08:12.964632 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e89abd-fe00-4c52-8dec-2bea95516a5c None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:12.965048 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e89abd-fe00-4c52-8dec-2bea95516a5c None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:12.965215 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e89abd-fe00-4c52-8dec-2bea95516a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:12.991106 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-85e89abd-fe00-4c52-8dec-2bea95516a5c None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:12.992332 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-85e89abd-fe00-4c52-8dec-2bea95516a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:13.066858 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 50/99] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:13.088674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7264e03-6b60-4cd9-8ddf-63c6fb4606a5 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:13.099107 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ae607-b063-4e08-9f96-1813fb980e85 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:13.099400 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ae607-b063-4e08-9f96-1813fb980e85 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:13.099668 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ae607-b063-4e08-9f96-1813fb980e85 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:13.101070 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 51/100] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:13.106892 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6b3c4-a5d6-4151-aaca-5c651a88404a None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:13.107269 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6b3c4-a5d6-4151-aaca-5c651a88404a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:13.107559 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6b3c4-a5d6-4151-aaca-5c651a88404a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:13.127303 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-b4b6b3c4-a5d6-4151-aaca-5c651a88404a None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:13.129094 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b4b6b3c4-a5d6-4151-aaca-5c651a88404a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:13.142402 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:13.219350 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7264e03-6b60-4cd9-8ddf-63c6fb4606a5 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:13.224311 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 52/101] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:13.246762 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6b3c4-a5d6-4151-aaca-5c651a88404a None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:13.286727 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb35a6a-12fc-4f2e-9118-18c7006a220a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:13.287407 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb35a6a-12fc-4f2e-9118-18c7006a220a None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:13.287696 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb35a6a-12fc-4f2e-9118-18c7006a220a None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:13.287973 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb35a6a-12fc-4f2e-9118-18c7006a220a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:13.303390 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb35a6a-12fc-4f2e-9118-18c7006a220a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:13.317325 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-ddb35a6a-12fc-4f2e-9118-18c7006a220a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:13.317325 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:13.317325 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:13.317325 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:13.317325 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:13.317325 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:13.317325 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:13.317325 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:13.317325 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:13.317325 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:13.317325 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:13.317325 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:13.317325 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:13.317325 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:13.317325 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:13.317325 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:13.318221 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:13.318221 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:13.318221 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:13.318221 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:13.318221 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:13.318221 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:13.318221 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:13.318221 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:13.318221 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:13.318221 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:13.318221 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:13.318221 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:13.318221 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:13.318221 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:13.318221 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:13.318221 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:13.319621 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:13.319621 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:13.319621 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:13.319621 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:13.319621 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:13.319621 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:13.319621 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:13.319621 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:13.319621 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:13.319621 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:13.319621 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:13.319621 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:08:13.319621 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 50/102] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:13.326868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb35a6a-12fc-4f2e-9118-18c7006a220a None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:13.334215 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:13.357370 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6b3c4-a5d6-4151-aaca-5c651a88404a None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:13.394018 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec059c2d-7348-4cb8-b7c3-3755dcec7ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:13.394784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec059c2d-7348-4cb8-b7c3-3755dcec7ceb None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:13.395071 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec059c2d-7348-4cb8-b7c3-3755dcec7ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:13.395833 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec059c2d-7348-4cb8-b7c3-3755dcec7ceb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:13.407059 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec059c2d-7348-4cb8-b7c3-3755dcec7ceb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:13.413715 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-ec059c2d-7348-4cb8-b7c3-3755dcec7ceb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:08:13.413715 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:13.413715 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:13.413715 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:13.413715 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:13.413715 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:13.413715 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:13.413715 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:13.413715 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:13.413715 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:13.413715 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:13.413715 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:13.413715 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:13.413715 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:13.413715 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:13.413715 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:13.414734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:13.414734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:13.414734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:13.414734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:13.414734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:13.414734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:13.414734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:13.414734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:13.414734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:13.414734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:13.414734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:13.414734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:13.414734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:13.414734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:13.414734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:13.414734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:13.415508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:13.415508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:13.415508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:13.415508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:13.415508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:13.415508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:13.415508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:13.415508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:13.415508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:13.415508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:13.415508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:08:13.415508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:08:13.416406 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 53/103] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:08:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:13.421804 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec059c2d-7348-4cb8-b7c3-3755dcec7ceb None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:13.442351 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:13.508995 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb35a6a-12fc-4f2e-9118-18c7006a220a None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:13.574698 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02633be9-aaaf-4b23-936e-75bcd4386389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:13.575350 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02633be9-aaaf-4b23-936e-75bcd4386389 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:13.575643 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02633be9-aaaf-4b23-936e-75bcd4386389 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:13.575929 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02633be9-aaaf-4b23-936e-75bcd4386389 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:13.576674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02633be9-aaaf-4b23-936e-75bcd4386389 None admin] RBAC: Adding query filter params (name=member) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:13.576984 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02633be9-aaaf-4b23-936e-75bcd4386389 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:13.578425 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02633be9-aaaf-4b23-936e-75bcd4386389 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:13.587931 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 51/104] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 27 04:08:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:13.607651 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02633be9-aaaf-4b23-936e-75bcd4386389 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:13.614576 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec059c2d-7348-4cb8-b7c3-3755dcec7ceb None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:13.650723 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540224d-79cc-424a-8543-a678aae39277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:13.651409 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540224d-79cc-424a-8543-a678aae39277 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:13.651709 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540224d-79cc-424a-8543-a678aae39277 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:13.652104 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540224d-79cc-424a-8543-a678aae39277 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:13.652833 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540224d-79cc-424a-8543-a678aae39277 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:13.653148 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540224d-79cc-424a-8543-a678aae39277 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:13.654593 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540224d-79cc-424a-8543-a678aae39277 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:13.663885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 54/105] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:13.669243 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540224d-79cc-424a-8543-a678aae39277 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:13.761324 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:13.779224 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:13.844165 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02633be9-aaaf-4b23-936e-75bcd4386389 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:13.844352 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540224d-79cc-424a-8543-a678aae39277 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:13.886366 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c7b2e5-4f87-4814-81d1-cd1093237db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:13.887136 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c7b2e5-4f87-4814-81d1-cd1093237db1 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:13.887424 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c7b2e5-4f87-4814-81d1-cd1093237db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:13.887703 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c7b2e5-4f87-4814-81d1-cd1093237db1 None admin] PATH_INFO: `/v3/users/32924f9a30fb4e7b835bb29d116319d9` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:13.889006 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c7b2e5-4f87-4814-81d1-cd1093237db1 None admin] RBAC: Authorizing `identity:get_user(user_id=32924f9a30fb4e7b835bb29d116319d9)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:13.890501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c7b2e5-4f87-4814-81d1-cd1093237db1 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:13.892632 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 55/106] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:13 2022] GET /identity/v3/users/32924f9a30fb4e7b835bb29d116319d9 => generated 247 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:13.898901 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c7b2e5-4f87-4814-81d1-cd1093237db1 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:13.926068 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5012a-805a-4629-826e-46004d024750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:13.926737 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5012a-805a-4629-826e-46004d024750 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:13.927019 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5012a-805a-4629-826e-46004d024750 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:13.927295 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5012a-805a-4629-826e-46004d024750 None admin] PATH_INFO: `/v3/users/706e785a4bca46e7ab7003c417e51cc1` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:13.928473 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c5012a-805a-4629-826e-46004d024750 None admin] RBAC: Authorizing `identity:get_user(user_id=706e785a4bca46e7ab7003c417e51cc1)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:13.929928 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c5012a-805a-4629-826e-46004d024750 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:13.940311 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 52/107] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:13 2022] GET /identity/v3/users/706e785a4bca46e7ab7003c417e51cc1 => generated 275 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:08:13.955000 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5012a-805a-4629-826e-46004d024750 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:14.009043 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c7b2e5-4f87-4814-81d1-cd1093237db1 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:14.045956 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d0bb3a-8207-4c29-88ae-0e644da532d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:14.046540 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d0bb3a-8207-4c29-88ae-0e644da532d2 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:14.046831 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d0bb3a-8207-4c29-88ae-0e644da532d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:14.047111 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d0bb3a-8207-4c29-88ae-0e644da532d2 None admin] PATH_INFO: `/v3/projects/85168a036bf94350836ae48489878ed3` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:14.048411 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d0bb3a-8207-4c29-88ae-0e644da532d2 None admin] RBAC: Authorizing `identity:get_project(project_id=85168a036bf94350836ae48489878ed3)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:14.050208 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d0bb3a-8207-4c29-88ae-0e644da532d2 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:14.052276 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 56/108] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:13 2022] GET /identity/v3/projects/85168a036bf94350836ae48489878ed3 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:14.058122 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d0bb3a-8207-4c29-88ae-0e644da532d2 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:14.070338 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:14.165930 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d0bb3a-8207-4c29-88ae-0e644da532d2 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:14.201210 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4410f56f-920a-4dd6-822e-5c672c530d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:14.201894 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410f56f-920a-4dd6-822e-5c672c530d6b None admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:14.202216 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410f56f-920a-4dd6-822e-5c672c530d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:14.202495 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410f56f-920a-4dd6-822e-5c672c530d6b None admin] PATH_INFO: `/v3/projects/85168a036bf94350836ae48489878ed3/users/32924f9a30fb4e7b835bb29d116319d9/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:14.204582 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4410f56f-920a-4dd6-822e-5c672c530d6b None admin] RBAC: Authorizing `identity:create_grant(project_id=85168a036bf94350836ae48489878ed3, user_id=32924f9a30fb4e7b835bb29d116319d9, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:14.206117 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4410f56f-920a-4dd6-822e-5c672c530d6b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:14.212233 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5012a-805a-4629-826e-46004d024750 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:14.229218 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 57/109] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 27 04:08:14 2022] PUT /identity/v3/projects/85168a036bf94350836ae48489878ed3/users/32924f9a30fb4e7b835bb29d116319d9/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:08:14.269520 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5770370-2a2a-475e-915a-db8722e848c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:14.270494 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5770370-2a2a-475e-915a-db8722e848c1 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:14.270937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5770370-2a2a-475e-915a-db8722e848c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:14.271350 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5770370-2a2a-475e-915a-db8722e848c1 None admin] PATH_INFO: `/v3/projects/85168a036bf94350836ae48489878ed3` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:14.272645 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5770370-2a2a-475e-915a-db8722e848c1 None admin] RBAC: Authorizing `identity:get_project(project_id=85168a036bf94350836ae48489878ed3)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:14.274516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5770370-2a2a-475e-915a-db8722e848c1 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:14.276387 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 53/110] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:13 2022] GET /identity/v3/projects/85168a036bf94350836ae48489878ed3 => generated 298 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:08:14.282447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4410f56f-920a-4dd6-822e-5c672c530d6b None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:14.305816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8033d9a4-da10-4f55-8960-e2b9b42030f7 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:14.306399 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8033d9a4-da10-4f55-8960-e2b9b42030f7 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:14.306822 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8033d9a4-da10-4f55-8960-e2b9b42030f7 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:14.308371 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 54/111] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 04:08:14.314304 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855a022-82f1-41db-9969-f774b77955bc None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:14.314775 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855a022-82f1-41db-9969-f774b77955bc None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:14.315747 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855a022-82f1-41db-9969-f774b77955bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:14.325000 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:14.349850 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-0855a022-82f1-41db-9969-f774b77955bc None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:14.351992 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0855a022-82f1-41db-9969-f774b77955bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:14.439675 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:14.486220 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 55/112] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 04:08:14.527558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855a022-82f1-41db-9969-f774b77955bc None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:14.533643 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4410f56f-920a-4dd6-822e-5c672c530d6b None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:14.609606 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adcef3a-14e3-4255-ab5c-99172e6a9224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:14.610584 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcef3a-14e3-4255-ab5c-99172e6a9224 None admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:14.610874 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcef3a-14e3-4255-ab5c-99172e6a9224 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:14.611563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcef3a-14e3-4255-ab5c-99172e6a9224 None admin] PATH_INFO: `/v3/projects/85168a036bf94350836ae48489878ed3/users/706e785a4bca46e7ab7003c417e51cc1/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:14.613882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adcef3a-14e3-4255-ab5c-99172e6a9224 None admin] RBAC: Authorizing `identity:create_grant(project_id=85168a036bf94350836ae48489878ed3, user_id=706e785a4bca46e7ab7003c417e51cc1, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:14.616261 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adcef3a-14e3-4255-ab5c-99172e6a9224 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:14.637108 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 58/113] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 27 04:08:14 2022] PUT /identity/v3/projects/85168a036bf94350836ae48489878ed3/users/706e785a4bca46e7ab7003c417e51cc1/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:08:14.639122 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6548c5-8045-4916-8c4b-5338b9cf1de3 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:14.639606 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6548c5-8045-4916-8c4b-5338b9cf1de3 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:14.639887 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6548c5-8045-4916-8c4b-5338b9cf1de3 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:14.641425 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 59/114] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:14.648666 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d3ed1-0ff8-4e92-b361-cc5785b050c2 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:14.648951 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d3ed1-0ff8-4e92-b361-cc5785b050c2 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:14.649497 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d3ed1-0ff8-4e92-b361-cc5785b050c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:14.664893 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855a022-82f1-41db-9969-f774b77955bc None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:14.673557 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-285d3ed1-0ff8-4e92-b361-cc5785b050c2 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:14.675607 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-285d3ed1-0ff8-4e92-b361-cc5785b050c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:14.703284 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:14.711303 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b53ef-0a58-4821-99f0-34428a32afe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:14.711938 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b53ef-0a58-4821-99f0-34428a32afe4 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:14.712221 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b53ef-0a58-4821-99f0-34428a32afe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:14.712606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b53ef-0a58-4821-99f0-34428a32afe4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:14.726708 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b53ef-0a58-4821-99f0-34428a32afe4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:14.732292 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-d38b53ef-0a58-4821-99f0-34428a32afe4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 04:08:14.732292 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:14.732292 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:14.732292 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:14.732292 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:14.732292 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:14.732292 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:14.732292 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:14.732292 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:14.732292 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:14.732292 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:14.732292 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:14.732292 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:14.732292 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:14.732292 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:14.732292 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:14.733194 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:14.733194 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:14.733194 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:14.733194 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:14.733194 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:14.733194 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:14.733194 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:14.733194 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:14.733194 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:14.733194 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:14.733194 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:14.733194 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:14.733194 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:14.733194 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:14.733194 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:14.733194 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:14.733939 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:14.733939 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:14.733939 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:14.733939 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:14.733939 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:14.733939 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:14.733939 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:14.733939 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:14.733939 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:14.733939 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:14.733939 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 04:08:14.733939 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:08:14.738099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 56/115] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:08:14.740532 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b53ef-0a58-4821-99f0-34428a32afe4 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:14.780424 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 60/116] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 04:08:14.804027 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d3ed1-0ff8-4e92-b361-cc5785b050c2 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:14.873003 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:14.880372 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b53ef-0a58-4821-99f0-34428a32afe4 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:14.921747 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d3ed1-0ff8-4e92-b361-cc5785b050c2 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:14.938825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f12659-4453-473d-957b-17380078b970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:14.939901 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f12659-4453-473d-957b-17380078b970 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:14.941068 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f12659-4453-473d-957b-17380078b970 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:14.941545 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f12659-4453-473d-957b-17380078b970 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:14.942632 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f12659-4453-473d-957b-17380078b970 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:14.943189 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f12659-4453-473d-957b-17380078b970 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:14.944797 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f12659-4453-473d-957b-17380078b970 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:14.956091 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 57/117] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 27 04:08:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:14.957862 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc890697-adea-44e0-a6eb-be818a62aa1d None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:14.958402 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc890697-adea-44e0-a6eb-be818a62aa1d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:14.958897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc890697-adea-44e0-a6eb-be818a62aa1d None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:14.960487 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 58/118] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:14.965655 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c742187-edc9-4427-85dd-f9e02b51db72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:14.966377 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c742187-edc9-4427-85dd-f9e02b51db72 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:14.966667 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c742187-edc9-4427-85dd-f9e02b51db72 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:14.966951 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c742187-edc9-4427-85dd-f9e02b51db72 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:14.967283 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb2994-68cd-4ab9-95a9-6717bad04ee1 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:14.967764 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb2994-68cd-4ab9-95a9-6717bad04ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:14.968187 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c742187-edc9-4427-85dd-f9e02b51db72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:14.968452 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb2994-68cd-4ab9-95a9-6717bad04ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:14.969637 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c742187-edc9-4427-85dd-f9e02b51db72 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:14.972937 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 61/119] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:08:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:08:14.981936 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c742187-edc9-4427-85dd-f9e02b51db72 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:14.998227 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-f3fb2994-68cd-4ab9-95a9-6717bad04ee1 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:14.999796 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f3fb2994-68cd-4ab9-95a9-6717bad04ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:15.059943 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:15.081930 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 59/120] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 04:08:15.083260 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb2994-68cd-4ab9-95a9-6717bad04ee1 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:15.099004 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c742187-edc9-4427-85dd-f9e02b51db72 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:15.130584 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:15.145934 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa10e2-4c85-40a1-8d5e-c60f3ffd26d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:15.147098 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa10e2-4c85-40a1-8d5e-c60f3ffd26d1 None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:15.147547 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa10e2-4c85-40a1-8d5e-c60f3ffd26d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:15.148299 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa10e2-4c85-40a1-8d5e-c60f3ffd26d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:15.149851 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa10e2-4c85-40a1-8d5e-c60f3ffd26d1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:15.151524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa10e2-4c85-40a1-8d5e-c60f3ffd26d1 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:15.190341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 62/121] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Jul 27 04:08:14 2022] POST /identity/v3/projects => generated 302 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:08:15.192504 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d1a5f-b231-409c-ad68-5f0fe6c03b32 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:15.192948 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d1a5f-b231-409c-ad68-5f0fe6c03b32 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:15.193789 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d1a5f-b231-409c-ad68-5f0fe6c03b32 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:15.195400 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 63/122] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:15.196375 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d1a5f-b231-409c-ad68-5f0fe6c03b32 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:15.231916 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb2994-68cd-4ab9-95a9-6717bad04ee1 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:15.306727 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef8ba8-572a-479a-9829-97438be1e0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:15.307418 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef8ba8-572a-479a-9829-97438be1e0ae None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:15.307700 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef8ba8-572a-479a-9829-97438be1e0ae None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:15.308048 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef8ba8-572a-479a-9829-97438be1e0ae None admin] PATH_INFO: `/v3/users/853f632e24be4c118a3ee00e5a2da179` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:15.344555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d1a5f-b231-409c-ad68-5f0fe6c03b32 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:15.347613 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef8ba8-572a-479a-9829-97438be1e0ae None admin] RBAC: Authorizing `identity:get_user(user_id=853f632e24be4c118a3ee00e5a2da179)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:15.349430 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef8ba8-572a-479a-9829-97438be1e0ae None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:15.351285 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 60/123] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:15 2022] GET /identity/v3/users/853f632e24be4c118a3ee00e5a2da179 => generated 289 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:15.352570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ecffdc-7205-4dae-a407-e194421b182c None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:15.352859 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ecffdc-7205-4dae-a407-e194421b182c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:15.353134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ecffdc-7205-4dae-a407-e194421b182c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:15.377914 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-49ecffdc-7205-4dae-a407-e194421b182c None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:15.379302 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-49ecffdc-7205-4dae-a407-e194421b182c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:15.389560 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5334ca-761b-451e-87f6-2c1995b8ca65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:15.390342 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5334ca-761b-451e-87f6-2c1995b8ca65 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:15.390734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5334ca-761b-451e-87f6-2c1995b8ca65 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:15.391373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5334ca-761b-451e-87f6-2c1995b8ca65 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:15.404512 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5334ca-761b-451e-87f6-2c1995b8ca65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:15.411484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-7b5334ca-761b-451e-87f6-2c1995b8ca65 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 04:08:15.411484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:15.411484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:15.411484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:15.411484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:15.411484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:15.411484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:15.411484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:15.411484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:15.411484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:15.411484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:15.411484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:15.411484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:15.411484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:15.411484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:15.411484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:15.412547 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:15.412547 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:15.412547 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:15.412547 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:15.412547 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:15.412547 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:15.412547 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:15.412547 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:15.412547 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:15.412547 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:15.412547 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:15.412547 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:15.412547 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:15.412547 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:15.412547 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:15.412547 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:15.413501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:15.413501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:15.413501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:15.413501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:15.413501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:15.413501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:15.413501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:15.413501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:15.413501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:15.413501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:15.413501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 04:08:15.413501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:08:15.415308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 64/124] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 27 04:08:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:15.416412 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5334ca-761b-451e-87f6-2c1995b8ca65 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:15.427404 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:15.462371 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:15.470936 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 61/125] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:15.473320 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ecffdc-7205-4dae-a407-e194421b182c None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:15.551208 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5334ca-761b-451e-87f6-2c1995b8ca65 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:15.591562 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3ec3e-27ee-42c0-913f-2d3a94fe9e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:15.592229 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3ec3e-27ee-42c0-913f-2d3a94fe9e8f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:15.592511 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3ec3e-27ee-42c0-913f-2d3a94fe9e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:15.592785 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3ec3e-27ee-42c0-913f-2d3a94fe9e8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:15.610594 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ecffdc-7205-4dae-a407-e194421b182c None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:15.671958 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72bfa3-18c0-4e99-879c-9b556e3aec2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:15.672517 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72bfa3-18c0-4e99-879c-9b556e3aec2d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:15.672801 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72bfa3-18c0-4e99-879c-9b556e3aec2d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:15.673079 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72bfa3-18c0-4e99-879c-9b556e3aec2d None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:15.673819 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72bfa3-18c0-4e99-879c-9b556e3aec2d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:15.674168 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72bfa3-18c0-4e99-879c-9b556e3aec2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:15.676467 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72bfa3-18c0-4e99-879c-9b556e3aec2d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:15.689279 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3ec3e-27ee-42c0-913f-2d3a94fe9e8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:15.691250 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3ec3e-27ee-42c0-913f-2d3a94fe9e8f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:15.695009 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 62/126] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:08:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:15.695870 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72bfa3-18c0-4e99-879c-9b556e3aec2d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:15.754658 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:15.761146 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:15.850684 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72bfa3-18c0-4e99-879c-9b556e3aec2d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:15.907863 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 65/127] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:08:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:08:15.909268 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3ec3e-27ee-42c0-913f-2d3a94fe9e8f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:15.917405 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2e7dd-50c3-4829-8cf8-7f0e9900924d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:15.920695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2e7dd-50c3-4829-8cf8-7f0e9900924d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:15.920986 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2e7dd-50c3-4829-8cf8-7f0e9900924d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:15.921264 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2e7dd-50c3-4829-8cf8-7f0e9900924d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:15.942288 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d2e7dd-50c3-4829-8cf8-7f0e9900924d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:15.947565 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-10d2e7dd-50c3-4829-8cf8-7f0e9900924d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:15.947565 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:15.947565 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:15.947565 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:15.947565 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:15.947565 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:15.947565 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:15.947565 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:15.947565 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:15.947565 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:15.947565 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:15.947565 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:15.947565 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:15.947565 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:15.947565 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:15.947565 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:15.948419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:15.948419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:15.948419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:15.948419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:15.948419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:15.948419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:15.948419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:15.948419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:15.948419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:15.948419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:15.948419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:15.948419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:15.948419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:15.948419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:15.948419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:15.948419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:15.949578 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:15.949578 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:15.949578 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:15.949578 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:15.949578 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:15.949578 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:15.949578 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:15.949578 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:15.949578 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:15.949578 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:15.949578 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:15.949578 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:08:15.949578 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 63/128] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:15.949578 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2e7dd-50c3-4829-8cf8-7f0e9900924d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:16.047721 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3ec3e-27ee-42c0-913f-2d3a94fe9e8f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:16.083367 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2938dcaf-cb2c-4b46-8d5f-0eb13bdfb639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:16.084027 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2938dcaf-cb2c-4b46-8d5f-0eb13bdfb639 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:16.084305 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2938dcaf-cb2c-4b46-8d5f-0eb13bdfb639 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:16.084583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2938dcaf-cb2c-4b46-8d5f-0eb13bdfb639 None admin] PATH_INFO: `/v3/users/706e785a4bca46e7ab7003c417e51cc1` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:16.085804 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2938dcaf-cb2c-4b46-8d5f-0eb13bdfb639 None admin] RBAC: Authorizing `identity:get_user(user_id=706e785a4bca46e7ab7003c417e51cc1)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:16.087326 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2938dcaf-cb2c-4b46-8d5f-0eb13bdfb639 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:16.089215 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 66/129] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:15 2022] GET /identity/v3/users/706e785a4bca46e7ab7003c417e51cc1 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:16.090629 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2938dcaf-cb2c-4b46-8d5f-0eb13bdfb639 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:16.097392 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:16.133326 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:16.150276 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2e7dd-50c3-4829-8cf8-7f0e9900924d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:16.203856 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2938dcaf-cb2c-4b46-8d5f-0eb13bdfb639 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:16.227287 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ef215-9d3c-476d-98d8-b1bb1af09ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:16.228361 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ef215-9d3c-476d-98d8-b1bb1af09ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:16.228861 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ef215-9d3c-476d-98d8-b1bb1af09ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:16.229337 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ef215-9d3c-476d-98d8-b1bb1af09ca7 None admin] PATH_INFO: `/v3/projects/85168a036bf94350836ae48489878ed3` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:16.234712 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ef215-9d3c-476d-98d8-b1bb1af09ca7 None admin] RBAC: Authorizing `identity:get_project(project_id=85168a036bf94350836ae48489878ed3)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:16.236867 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ef215-9d3c-476d-98d8-b1bb1af09ca7 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:16.238827 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 64/130] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:15 2022] GET /identity/v3/projects/85168a036bf94350836ae48489878ed3 => generated 298 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:16.239936 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ef215-9d3c-476d-98d8-b1bb1af09ca7 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:16.243372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4f2bf-a88f-4c9a-83ac-b5c6b1549708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:16.244441 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4f2bf-a88f-4c9a-83ac-b5c6b1549708 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:16.244729 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4f2bf-a88f-4c9a-83ac-b5c6b1549708 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:16.245007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4f2bf-a88f-4c9a-83ac-b5c6b1549708 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:16.245742 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb4f2bf-a88f-4c9a-83ac-b5c6b1549708 None admin] RBAC: Adding query filter params (name=member) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:16.246070 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb4f2bf-a88f-4c9a-83ac-b5c6b1549708 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:16.247464 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb4f2bf-a88f-4c9a-83ac-b5c6b1549708 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:16.262666 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 67/131] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 27 04:08:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:16.262666 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4f2bf-a88f-4c9a-83ac-b5c6b1549708 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:16.343916 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:16.377949 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ef215-9d3c-476d-98d8-b1bb1af09ca7 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:16.414120 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af775c16-04af-40d7-9589-5c671e06ac61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:16.415153 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af775c16-04af-40d7-9589-5c671e06ac61 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:16.415660 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af775c16-04af-40d7-9589-5c671e06ac61 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:16.416145 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af775c16-04af-40d7-9589-5c671e06ac61 None admin] PATH_INFO: `/v3/projects/85168a036bf94350836ae48489878ed3` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:16.417519 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af775c16-04af-40d7-9589-5c671e06ac61 None admin] RBAC: Authorizing `identity:get_project(project_id=85168a036bf94350836ae48489878ed3)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:16.419298 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af775c16-04af-40d7-9589-5c671e06ac61 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:16.421177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 65/132] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:16 2022] GET /identity/v3/projects/85168a036bf94350836ae48489878ed3 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:16.422324 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af775c16-04af-40d7-9589-5c671e06ac61 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:16.458649 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:16.499938 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4f2bf-a88f-4c9a-83ac-b5c6b1549708 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:16.552815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af775c16-04af-40d7-9589-5c671e06ac61 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:16.574718 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee9947-1283-490e-9caa-b8b74e0be643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:16.575717 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee9947-1283-490e-9caa-b8b74e0be643 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:16.576201 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee9947-1283-490e-9caa-b8b74e0be643 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:16.576694 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee9947-1283-490e-9caa-b8b74e0be643 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:16.577709 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee9947-1283-490e-9caa-b8b74e0be643 None admin] RBAC: Adding query filter params (role.id=84fc927928374b91847a13b7fbb8ddd1, scope.project.id=85168a036bf94350836ae48489878ed3, user.id=853f632e24be4c118a3ee00e5a2da179) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:16.578277 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee9947-1283-490e-9caa-b8b74e0be643 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:16.579949 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee9947-1283-490e-9caa-b8b74e0be643 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:16.594547 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 68/133] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:08:16 2022] GET /identity/v3/role_assignments?user.id=853f632e24be4c118a3ee00e5a2da179&scope.project.id=85168a036bf94350836ae48489878ed3&role.id=84fc927928374b91847a13b7fbb8ddd1 => generated 259 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:16.595789 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee9947-1283-490e-9caa-b8b74e0be643 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:16.623823 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7f97d-5b5c-4fdd-b2da-26f14e2da95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:16.625844 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7f97d-5b5c-4fdd-b2da-26f14e2da95a None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:16.628257 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7f97d-5b5c-4fdd-b2da-26f14e2da95a None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:16.628785 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7f97d-5b5c-4fdd-b2da-26f14e2da95a None admin] PATH_INFO: `/v3/users/706e785a4bca46e7ab7003c417e51cc1` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:16.631384 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d7f97d-5b5c-4fdd-b2da-26f14e2da95a None admin] RBAC: Authorizing `identity:get_user(user_id=706e785a4bca46e7ab7003c417e51cc1)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:16.633460 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d7f97d-5b5c-4fdd-b2da-26f14e2da95a None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:16.636876 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 66/134] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:16 2022] GET /identity/v3/users/706e785a4bca46e7ab7003c417e51cc1 => generated 275 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:16.642554 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7f97d-5b5c-4fdd-b2da-26f14e2da95a None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:16.792825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7f97d-5b5c-4fdd-b2da-26f14e2da95a None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:16.809277 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee9947-1283-490e-9caa-b8b74e0be643 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:16.817010 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:16.835349 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d92db-2cd4-4398-9f71-89ef108f4d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:16.836395 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d92db-2cd4-4398-9f71-89ef108f4d3d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:16.836897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d92db-2cd4-4398-9f71-89ef108f4d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:16.837399 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d92db-2cd4-4398-9f71-89ef108f4d3d None admin] PATH_INFO: `/v3/projects/e6014d57b86c4cf1848ccb82dc88cea9` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:16.849084 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d92db-2cd4-4398-9f71-89ef108f4d3d None admin] RBAC: Authorizing `identity:get_project(project_id=e6014d57b86c4cf1848ccb82dc88cea9)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:16.851313 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d92db-2cd4-4398-9f71-89ef108f4d3d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:16.857355 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 67/135] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:16 2022] GET /identity/v3/projects/e6014d57b86c4cf1848ccb82dc88cea9 => generated 312 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:16.863099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d92db-2cd4-4398-9f71-89ef108f4d3d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:16.884662 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:16.905650 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2372d-3662-4428-8d7b-54ccadc42918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:16.906378 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2372d-3662-4428-8d7b-54ccadc42918 None admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:16.906682 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2372d-3662-4428-8d7b-54ccadc42918 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:16.906971 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2372d-3662-4428-8d7b-54ccadc42918 None admin] PATH_INFO: `/v3/projects/85168a036bf94350836ae48489878ed3/users/706e785a4bca46e7ab7003c417e51cc1/roles/cdfc87279ea24b65bae3961035133725` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:16.915640 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd2372d-3662-4428-8d7b-54ccadc42918 None admin] RBAC: Authorizing `identity:create_grant(project_id=85168a036bf94350836ae48489878ed3, user_id=706e785a4bca46e7ab7003c417e51cc1, role_id=cdfc87279ea24b65bae3961035133725)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:16.919169 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd2372d-3662-4428-8d7b-54ccadc42918 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:16.952437 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 69/136] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 27 04:08:16 2022] PUT /identity/v3/projects/85168a036bf94350836ae48489878ed3/users/706e785a4bca46e7ab7003c417e51cc1/roles/cdfc87279ea24b65bae3961035133725 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:08:17.006813 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d92db-2cd4-4398-9f71-89ef108f4d3d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:17.056395 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f0fb5d-f352-4127-85d8-823900a0764e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:17.057472 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0fb5d-f352-4127-85d8-823900a0764e None admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:17.057992 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0fb5d-f352-4127-85d8-823900a0764e None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:17.058525 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0fb5d-f352-4127-85d8-823900a0764e None admin] PATH_INFO: `/v3/projects/e6014d57b86c4cf1848ccb82dc88cea9/users/706e785a4bca46e7ab7003c417e51cc1/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:17.060808 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f0fb5d-f352-4127-85d8-823900a0764e None admin] RBAC: Authorizing `identity:create_grant(project_id=e6014d57b86c4cf1848ccb82dc88cea9, user_id=706e785a4bca46e7ab7003c417e51cc1, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:17.062563 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f0fb5d-f352-4127-85d8-823900a0764e None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:17.089295 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 68/137] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 27 04:08:16 2022] PUT /identity/v3/projects/e6014d57b86c4cf1848ccb82dc88cea9/users/706e785a4bca46e7ab7003c417e51cc1/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:08:17.222124 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e71fa-d5ff-42a9-bdf2-e290865a5583 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:17.223196 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e71fa-d5ff-42a9-bdf2-e290865a5583 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:17.223705 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e71fa-d5ff-42a9-bdf2-e290865a5583 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:17.225358 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 70/138] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:17.239473 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dbb88c-a6d8-41e7-8f3b-d4666775511e None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:17.240470 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dbb88c-a6d8-41e7-8f3b-d4666775511e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:17.241006 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dbb88c-a6d8-41e7-8f3b-d4666775511e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:17.263065 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-21dbb88c-a6d8-41e7-8f3b-d4666775511e None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:17.264690 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-21dbb88c-a6d8-41e7-8f3b-d4666775511e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:17.288952 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:17.349441 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 69/139] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:17.377545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e71fa-d5ff-42a9-bdf2-e290865a5583 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:17.439079 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fae70-6d50-43ed-8395-bd59b4a2c4d0 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:17.439717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fae70-6d50-43ed-8395-bd59b4a2c4d0 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:17.440988 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fae70-6d50-43ed-8395-bd59b4a2c4d0 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:17.444321 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 70/140] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 04:08:17.450543 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614b30d-1e76-4b93-a2a8-a173c887259e None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:17.451170 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614b30d-1e76-4b93-a2a8-a173c887259e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:17.451664 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614b30d-1e76-4b93-a2a8-a173c887259e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:17.476064 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-a614b30d-1e76-4b93-a2a8-a173c887259e None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:17.477696 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-a614b30d-1e76-4b93-a2a8-a173c887259e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:17.529936 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e71fa-d5ff-42a9-bdf2-e290865a5583 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:17.552009 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 71/141] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 04:08:17.577818 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a614b30d-1e76-4b93-a2a8-a173c887259e None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:17.593898 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569100e-d90c-4be7-88b0-8591abaa3641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:17.594599 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569100e-d90c-4be7-88b0-8591abaa3641 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:17.595112 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569100e-d90c-4be7-88b0-8591abaa3641 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:17.595217 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569100e-d90c-4be7-88b0-8591abaa3641 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:17.634696 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a569100e-d90c-4be7-88b0-8591abaa3641 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:17.640050 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-a569100e-d90c-4be7-88b0-8591abaa3641 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:08:17.640050 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:17.640050 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:17.640050 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:17.640050 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:17.640050 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:17.640050 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:17.640050 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:17.640050 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:17.640050 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:17.640050 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:17.640050 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:17.640050 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:17.640050 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:17.640050 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:17.640050 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:17.640953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:17.640953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:17.640953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:17.640953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:17.640953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:17.640953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:17.640953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:17.640953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:17.640953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:17.640953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:17.640953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:17.640953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:17.640953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:17.640953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:17.640953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:17.640953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:17.642182 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:17.642182 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:17.642182 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:17.642182 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:17.642182 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:17.642182 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:17.642182 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:17.642182 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:17.642182 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:17.642182 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:17.642182 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:08:17.642182 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:08:17.642182 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 71/142] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:08:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:17.648755 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569100e-d90c-4be7-88b0-8591abaa3641 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:17.686428 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:17.691936 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a614b30d-1e76-4b93-a2a8-a173c887259e None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:17.729187 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df7971-ab56-42a0-a693-a91ac98625dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:17.730279 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df7971-ab56-42a0-a693-a91ac98625dd None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:17.730804 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df7971-ab56-42a0-a693-a91ac98625dd None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:17.731296 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df7971-ab56-42a0-a693-a91ac98625dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:17.746441 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df7971-ab56-42a0-a693-a91ac98625dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:17.759667 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-80df7971-ab56-42a0-a693-a91ac98625dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:17.759667 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:17.759667 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:17.759667 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:17.759667 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:17.759667 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:17.759667 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:17.759667 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:17.759667 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:17.759667 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:17.759667 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:17.759667 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:17.759667 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:17.759667 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:17.759667 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:17.759667 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:17.760560 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:17.760560 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:17.760560 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:17.760560 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:17.760560 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:17.760560 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:17.760560 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:17.760560 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:17.760560 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:17.760560 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:17.760560 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:17.760560 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:17.760560 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:17.760560 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:17.760560 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:17.760560 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:17.761412 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:17.761412 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:17.761412 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:17.761412 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:17.761412 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:17.761412 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:17.761412 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:17.761412 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:17.761412 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:17.761412 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:17.761412 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:17.761412 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:08:17.765386 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 72/143] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:08:17.771688 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df7971-ab56-42a0-a693-a91ac98625dd None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:17.774347 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:17.867884 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569100e-d90c-4be7-88b0-8591abaa3641 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:17.889035 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df7971-ab56-42a0-a693-a91ac98625dd None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:17.925342 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd7787-c49a-4ef7-8328-69bb93c1b10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:17.926401 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd7787-c49a-4ef7-8328-69bb93c1b10a None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:17.926914 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd7787-c49a-4ef7-8328-69bb93c1b10a None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:17.927402 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd7787-c49a-4ef7-8328-69bb93c1b10a None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:17.928461 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd7787-c49a-4ef7-8328-69bb93c1b10a None admin] RBAC: Adding query filter params (name=member) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:17.928979 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd7787-c49a-4ef7-8328-69bb93c1b10a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:17.930642 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd7787-c49a-4ef7-8328-69bb93c1b10a None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:17.940927 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 73/144] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 27 04:08:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:08:17.946991 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd7787-c49a-4ef7-8328-69bb93c1b10a None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:17.960504 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:17.964290 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68ff5a-8243-4990-97f8-7fbf5e30b18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:17.964667 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68ff5a-8243-4990-97f8-7fbf5e30b18c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:17.964947 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68ff5a-8243-4990-97f8-7fbf5e30b18c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:17.965222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68ff5a-8243-4990-97f8-7fbf5e30b18c None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:17.966013 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68ff5a-8243-4990-97f8-7fbf5e30b18c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:17.966284 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68ff5a-8243-4990-97f8-7fbf5e30b18c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:17.967676 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68ff5a-8243-4990-97f8-7fbf5e30b18c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:17.985505 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 72/145] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:17.989704 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a77ef0-459e-4858-aa00-aaf32765e12f None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:17.990011 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a77ef0-459e-4858-aa00-aaf32765e12f None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:17.991838 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a77ef0-459e-4858-aa00-aaf32765e12f None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:17.992614 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 73/146] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:17.997720 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a77ef0-459e-4858-aa00-aaf32765e12f None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:18.098440 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd7787-c49a-4ef7-8328-69bb93c1b10a None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:18.167437 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4431fa7-07e1-4483-9b4b-b4afe418f3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:18.168029 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4431fa7-07e1-4483-9b4b-b4afe418f3a7 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:18.168318 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4431fa7-07e1-4483-9b4b-b4afe418f3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:18.168601 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4431fa7-07e1-4483-9b4b-b4afe418f3a7 None admin] PATH_INFO: `/v3/users/706e785a4bca46e7ab7003c417e51cc1` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:18.169988 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4431fa7-07e1-4483-9b4b-b4afe418f3a7 None admin] RBAC: Authorizing `identity:get_user(user_id=706e785a4bca46e7ab7003c417e51cc1)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:18.171460 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4431fa7-07e1-4483-9b4b-b4afe418f3a7 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:18.173088 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 74/147] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:17 2022] GET /identity/v3/users/706e785a4bca46e7ab7003c417e51cc1 => generated 275 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:08:18.181771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a431c22-81d7-4a28-a189-85788bf1272e None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:18.182079 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a431c22-81d7-4a28-a189-85788bf1272e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:18.182365 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a431c22-81d7-4a28-a189-85788bf1272e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:18.203958 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a77ef0-459e-4858-aa00-aaf32765e12f None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:18.217447 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-7a431c22-81d7-4a28-a189-85788bf1272e None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:18.218683 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-7a431c22-81d7-4a28-a189-85788bf1272e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:18.222852 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:18.272357 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b792c-a633-4a33-9c50-788dae3291c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:18.273024 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b792c-a633-4a33-9c50-788dae3291c6 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:18.275376 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b792c-a633-4a33-9c50-788dae3291c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:18.275680 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b792c-a633-4a33-9c50-788dae3291c6 None admin] PATH_INFO: `/v3/users/32924f9a30fb4e7b835bb29d116319d9` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:18.278095 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b792c-a633-4a33-9c50-788dae3291c6 None admin] RBAC: Authorizing `identity:get_user(user_id=32924f9a30fb4e7b835bb29d116319d9)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:18.281557 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b792c-a633-4a33-9c50-788dae3291c6 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:18.285138 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 74/148] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:17 2022] GET /identity/v3/users/32924f9a30fb4e7b835bb29d116319d9 => generated 247 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:18.286963 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b792c-a633-4a33-9c50-788dae3291c6 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:18.351225 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 75/149] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:18.353862 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a431c22-81d7-4a28-a189-85788bf1272e None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:18.368203 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:18.520311 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b792c-a633-4a33-9c50-788dae3291c6 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:18.559434 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a431c22-81d7-4a28-a189-85788bf1272e None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:18.571497 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7687d8a0-0bc1-4615-b595-b61380ca9aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:18.572140 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687d8a0-0bc1-4615-b595-b61380ca9aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:18.572424 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687d8a0-0bc1-4615-b595-b61380ca9aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:18.572700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7687d8a0-0bc1-4615-b595-b61380ca9aa8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:18.625411 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:18.650686 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2925c3-79b9-47e8-8629-f2593ce3a602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:18.651047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7687d8a0-0bc1-4615-b595-b61380ca9aa8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:18.651332 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2925c3-79b9-47e8-8629-f2593ce3a602 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:18.651620 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2925c3-79b9-47e8-8629-f2593ce3a602 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:18.651896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2925c3-79b9-47e8-8629-f2593ce3a602 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:18.652518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7687d8a0-0bc1-4615-b595-b61380ca9aa8 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:18.774787 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 75/150] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:08:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:08:18.780126 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7687d8a0-0bc1-4615-b595-b61380ca9aa8 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:18.822792 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2925c3-79b9-47e8-8629-f2593ce3a602 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:18.824620 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2925c3-79b9-47e8-8629-f2593ce3a602 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:18.842328 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:18.869757 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:18.895610 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7687d8a0-0bc1-4615-b595-b61380ca9aa8 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:18.942126 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037af85-d871-4e38-bcaf-bddc01758dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:18.942789 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037af85-d871-4e38-bcaf-bddc01758dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:18.943070 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037af85-d871-4e38-bcaf-bddc01758dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:18.943342 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037af85-d871-4e38-bcaf-bddc01758dc0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:18.945326 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4037af85-d871-4e38-bcaf-bddc01758dc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:18.946835 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4037af85-d871-4e38-bcaf-bddc01758dc0 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:18.948438 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 76/151] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:08:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:18.952020 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037af85-d871-4e38-bcaf-bddc01758dc0 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:19.073490 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 76/152] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:08:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 722 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:08:19.074697 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2925c3-79b9-47e8-8629-f2593ce3a602 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:19.079273 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037af85-d871-4e38-bcaf-bddc01758dc0 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:19.121369 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce07cd6-5e90-455a-a8ce-6829ad6e06ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:19.122055 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce07cd6-5e90-455a-a8ce-6829ad6e06ec None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:19.123780 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce07cd6-5e90-455a-a8ce-6829ad6e06ec None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:19.124058 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce07cd6-5e90-455a-a8ce-6829ad6e06ec None admin] PATH_INFO: `/v3/projects/85168a036bf94350836ae48489878ed3` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:19.128835 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce07cd6-5e90-455a-a8ce-6829ad6e06ec None admin] RBAC: Authorizing `identity:get_project(project_id=85168a036bf94350836ae48489878ed3)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:19.130340 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce07cd6-5e90-455a-a8ce-6829ad6e06ec None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:19.132216 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 77/153] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:18 2022] GET /identity/v3/projects/85168a036bf94350836ae48489878ed3 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:19.132803 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce07cd6-5e90-455a-a8ce-6829ad6e06ec None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:19.168364 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:19.249006 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2925c3-79b9-47e8-8629-f2593ce3a602 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:19.257999 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:19.287522 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0657e-bd8e-41ba-95ce-8319a0b45916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:19.288158 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0657e-bd8e-41ba-95ce-8319a0b45916 None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:19.288440 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0657e-bd8e-41ba-95ce-8319a0b45916 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:19.288719 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0657e-bd8e-41ba-95ce-8319a0b45916 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:19.289416 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f0657e-bd8e-41ba-95ce-8319a0b45916 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:19.290900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f0657e-bd8e-41ba-95ce-8319a0b45916 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:19.312785 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce07cd6-5e90-455a-a8ce-6829ad6e06ec None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:19.338524 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 77/154] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 27 04:08:19 2022] POST /identity/v3/users => generated 283 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:08:19.340277 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0657e-bd8e-41ba-95ce-8319a0b45916 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:19.395912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2640ec14-99e8-4611-a93e-26902c172f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:19.396598 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640ec14-99e8-4611-a93e-26902c172f95 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:19.396877 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640ec14-99e8-4611-a93e-26902c172f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:19.397270 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640ec14-99e8-4611-a93e-26902c172f95 None admin] PATH_INFO: `/v3/projects/85168a036bf94350836ae48489878ed3` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:19.398601 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2640ec14-99e8-4611-a93e-26902c172f95 None admin] RBAC: Authorizing `identity:get_project(project_id=85168a036bf94350836ae48489878ed3)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:19.400085 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2640ec14-99e8-4611-a93e-26902c172f95 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:19.402153 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 78/155] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:19 2022] GET /identity/v3/projects/85168a036bf94350836ae48489878ed3 => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:19.408436 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2640ec14-99e8-4611-a93e-26902c172f95 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:19.558772 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:19.582357 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2640ec14-99e8-4611-a93e-26902c172f95 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:19.604246 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0657e-bd8e-41ba-95ce-8319a0b45916 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:19.637158 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f724809b-b712-433e-90bd-6a954459d31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:19.637792 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f724809b-b712-433e-90bd-6a954459d31a None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:19.638100 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f724809b-b712-433e-90bd-6a954459d31a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:19.638383 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f724809b-b712-433e-90bd-6a954459d31a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:19.639300 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f724809b-b712-433e-90bd-6a954459d31a None admin] RBAC: Adding query filter params (role.id=cf771a8e94ee46149eb081347fea6d67, scope.project.id=85168a036bf94350836ae48489878ed3, user.id=32924f9a30fb4e7b835bb29d116319d9) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:19.639614 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f724809b-b712-433e-90bd-6a954459d31a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:19.641184 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f724809b-b712-433e-90bd-6a954459d31a None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:19.648770 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 79/156] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:08:19 2022] GET /identity/v3/role_assignments?user.id=32924f9a30fb4e7b835bb29d116319d9&scope.project.id=85168a036bf94350836ae48489878ed3&role.id=cf771a8e94ee46149eb081347fea6d67 => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:19.650002 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eaecbc-09cc-4d08-acde-eff66331ddd2 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:19.651950 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd3ca6-4b95-466d-8054-220cae7723df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:19.652684 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd3ca6-4b95-466d-8054-220cae7723df None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:19.652969 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd3ca6-4b95-466d-8054-220cae7723df None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:19.653247 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd3ca6-4b95-466d-8054-220cae7723df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:19.654457 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eaecbc-09cc-4d08-acde-eff66331ddd2 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:19.654750 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eaecbc-09cc-4d08-acde-eff66331ddd2 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:19.654962 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd3ca6-4b95-466d-8054-220cae7723df None admin] RBAC: Adding query filter params (role.id=7b6f2ade82434ee79175385de17049eb, scope.project.id=85168a036bf94350836ae48489878ed3, user.id=706e785a4bca46e7ab7003c417e51cc1) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:19.655288 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd3ca6-4b95-466d-8054-220cae7723df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:19.656155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 80/157] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:19.657185 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd3ca6-4b95-466d-8054-220cae7723df None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:19.662399 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd6618e-2efb-4971-8cb1-c910c2390fc3 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:19.662683 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd6618e-2efb-4971-8cb1-c910c2390fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:19.662960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd6618e-2efb-4971-8cb1-c910c2390fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:19.666581 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 78/158] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:08:19 2022] GET /identity/v3/role_assignments?user.id=706e785a4bca46e7ab7003c417e51cc1&scope.project.id=85168a036bf94350836ae48489878ed3&role.id=7b6f2ade82434ee79175385de17049eb => generated 608 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:19.693001 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-0cd6618e-2efb-4971-8cb1-c910c2390fc3 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:19.694325 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-0cd6618e-2efb-4971-8cb1-c910c2390fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:19.804348 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 81/159] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 04:08:19.835920 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1d833-fc79-41e7-9a18-40045681fad3 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:19.836305 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1d833-fc79-41e7-9a18-40045681fad3 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:19.836473 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1d833-fc79-41e7-9a18-40045681fad3 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:19.838807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd6618e-2efb-4971-8cb1-c910c2390fc3 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:19.841337 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 79/160] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 04:08:19.853862 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf91e8-22d1-4c41-b313-4d3ee19dd9f7 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:19.854309 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf91e8-22d1-4c41-b313-4d3ee19dd9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:19.854476 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf91e8-22d1-4c41-b313-4d3ee19dd9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:19.890874 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-d5cf91e8-22d1-4c41-b313-4d3ee19dd9f7 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:19.892154 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d5cf91e8-22d1-4c41-b313-4d3ee19dd9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:19.923546 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:19.963517 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:19.981272 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd6618e-2efb-4971-8cb1-c910c2390fc3 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:19.988784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 80/161] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 04:08:19.990276 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454a637-5ac0-4ba0-a497-89a1ad2de407 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:19.990577 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454a637-5ac0-4ba0-a497-89a1ad2de407 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:19.990856 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454a637-5ac0-4ba0-a497-89a1ad2de407 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:19.992130 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 81/162] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:19.999254 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1bddbc-0873-466e-bea6-ec738051e91d None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:19.999542 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1bddbc-0873-466e-bea6-ec738051e91d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:19.999829 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1bddbc-0873-466e-bea6-ec738051e91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:20.023304 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-2c1bddbc-0873-466e-bea6-ec738051e91d None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:20.024648 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2c1bddbc-0873-466e-bea6-ec738051e91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:20.056069 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88ae5d-8a17-4046-b0c8-cd92bb2ac1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:20.057084 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88ae5d-8a17-4046-b0c8-cd92bb2ac1e3 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:20.057608 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88ae5d-8a17-4046-b0c8-cd92bb2ac1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:20.058105 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88ae5d-8a17-4046-b0c8-cd92bb2ac1e3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:20.076909 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d88ae5d-8a17-4046-b0c8-cd92bb2ac1e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:20.083052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-1d88ae5d-8a17-4046-b0c8-cd92bb2ac1e3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 04:08:20.083052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:20.083052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:20.083052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:20.083052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:20.083052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:20.083052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:20.083052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:20.083052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:20.083052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:20.083052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:20.083052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:20.083052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:20.083052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:20.083052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:20.083052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:20.083912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:20.083912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:20.083912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:20.083912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:20.083912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:20.083912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:20.083912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:20.083912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:20.083912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:20.083912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:20.083912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:20.083912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:20.083912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:20.083912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:20.083912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:20.083912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:20.084761 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:20.084761 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:20.084761 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:20.084761 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:20.084761 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:20.084761 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:20.084761 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:20.084761 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:20.084761 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:20.084761 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:20.084761 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 04:08:20.084761 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:08:20.084761 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 82/163] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:08:20.085389 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88ae5d-8a17-4046-b0c8-cd92bb2ac1e3 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:20.106946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 82/164] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 04:08:20.107801 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1bddbc-0873-466e-bea6-ec738051e91d None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:20.214284 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88ae5d-8a17-4046-b0c8-cd92bb2ac1e3 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:20.235320 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:20.245872 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1bddbc-0873-466e-bea6-ec738051e91d None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:20.253714 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374b38b-1410-49f8-a022-17272be6a262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:20.254413 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374b38b-1410-49f8-a022-17272be6a262 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:20.254696 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374b38b-1410-49f8-a022-17272be6a262 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:20.255084 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374b38b-1410-49f8-a022-17272be6a262 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:20.262304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:20.269675 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374b38b-1410-49f8-a022-17272be6a262 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:20.275326 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-4374b38b-1410-49f8-a022-17272be6a262 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:20.275326 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:20.275326 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:20.275326 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:20.275326 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:20.275326 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:20.275326 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:20.275326 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:20.275326 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:20.275326 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:20.275326 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:20.275326 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:20.275326 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:20.275326 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:20.275326 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:20.275326 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:20.276183 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:20.276183 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:20.276183 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:20.276183 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:20.276183 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:20.276183 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:20.276183 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:20.276183 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:20.276183 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:20.276183 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:20.276183 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:20.276183 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:20.276183 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:20.276183 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:20.276183 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:20.276183 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:20.276960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:20.276960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:20.276960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:20.276960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:20.276960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:20.276960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:20.276960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:20.276960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:20.276960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:20.276960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:20.276960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:20.276960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:08:20.276960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 83/165] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:20.277829 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374b38b-1410-49f8-a022-17272be6a262 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:20.290595 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e5f88-0d8e-4831-a170-b9842b52f051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:20.291304 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e5f88-0d8e-4831-a170-b9842b52f051 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:20.291584 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e5f88-0d8e-4831-a170-b9842b52f051 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:20.291853 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e5f88-0d8e-4831-a170-b9842b52f051 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:20.292582 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e5f88-0d8e-4831-a170-b9842b52f051 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:20.292883 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e5f88-0d8e-4831-a170-b9842b52f051 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:20.294989 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e5f88-0d8e-4831-a170-b9842b52f051 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:20.303710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 83/166] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 27 04:08:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:20.304613 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e5f88-0d8e-4831-a170-b9842b52f051 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:20.400825 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374b38b-1410-49f8-a022-17272be6a262 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:20.422960 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e5f88-0d8e-4831-a170-b9842b52f051 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:20.442850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb6d8c-500d-4b96-89d8-8d8bd7e03ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:20.443921 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb6d8c-500d-4b96-89d8-8d8bd7e03ca5 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:20.444412 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb6d8c-500d-4b96-89d8-8d8bd7e03ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:20.444944 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb6d8c-500d-4b96-89d8-8d8bd7e03ca5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:20.462013 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb6d8c-500d-4b96-89d8-8d8bd7e03ca5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:20.468025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-3ceb6d8c-500d-4b96-89d8-8d8bd7e03ca5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 04:08:20.468025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:20.468025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:20.468025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:20.468025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:20.468025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:20.468025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:20.468025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:20.468025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:20.468025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:20.468025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:20.468025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:20.468025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:20.468025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:20.468025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:20.468025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:20.468953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:20.468953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:20.468953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:20.468953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:20.468953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:20.468953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:20.468953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:20.468953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:20.468953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:20.468953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:20.468953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:20.468953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:20.468953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:20.468953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:20.468953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:20.468953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:20.469806 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:20.469806 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:20.469806 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:20.469806 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:20.469806 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:20.469806 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:20.469806 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:20.469806 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:20.469806 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:20.469806 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:20.469806 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 04:08:20.469806 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:08:20.472584 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 84/167] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 27 04:08:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:20.473747 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb6d8c-500d-4b96-89d8-8d8bd7e03ca5 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:20.476071 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5996c36e-7a5a-4693-9d96-be75e07db6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:20.476806 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996c36e-7a5a-4693-9d96-be75e07db6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:20.477426 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996c36e-7a5a-4693-9d96-be75e07db6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:20.477771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996c36e-7a5a-4693-9d96-be75e07db6c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:20.479606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5996c36e-7a5a-4693-9d96-be75e07db6c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:20.479984 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5996c36e-7a5a-4693-9d96-be75e07db6c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:20.481428 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5996c36e-7a5a-4693-9d96-be75e07db6c9 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:20.492412 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 84/168] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 27 04:08:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:20.493752 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5996c36e-7a5a-4693-9d96-be75e07db6c9 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:20.513804 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:20.571557 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:20.652454 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb6d8c-500d-4b96-89d8-8d8bd7e03ca5 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:20.673634 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5996c36e-7a5a-4693-9d96-be75e07db6c9 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:20.720911 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd367474-ff07-4dff-b40d-969698b470e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:20.722062 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd367474-ff07-4dff-b40d-969698b470e8 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:20.722530 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd367474-ff07-4dff-b40d-969698b470e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:20.722990 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd367474-ff07-4dff-b40d-969698b470e8 None admin] PATH_INFO: `/v3/users/853f632e24be4c118a3ee00e5a2da179` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:20.724440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd367474-ff07-4dff-b40d-969698b470e8 None admin] RBAC: Authorizing `identity:get_user(user_id=853f632e24be4c118a3ee00e5a2da179)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:20.726198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd367474-ff07-4dff-b40d-969698b470e8 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:20.728600 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 85/169] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:20 2022] GET /identity/v3/users/853f632e24be4c118a3ee00e5a2da179 => generated 289 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:20.730401 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd367474-ff07-4dff-b40d-969698b470e8 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:20.737583 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2af72e-51ca-4430-8327-7ed1467225f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:20.738540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2af72e-51ca-4430-8327-7ed1467225f8 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:20.739068 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2af72e-51ca-4430-8327-7ed1467225f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:20.739522 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2af72e-51ca-4430-8327-7ed1467225f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:20.740429 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2af72e-51ca-4430-8327-7ed1467225f8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:20.742685 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2af72e-51ca-4430-8327-7ed1467225f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:20.745123 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2af72e-51ca-4430-8327-7ed1467225f8 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:20.761245 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 85/170] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:08:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:20.763282 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2af72e-51ca-4430-8327-7ed1467225f8 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:20.851087 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:20.905930 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2af72e-51ca-4430-8327-7ed1467225f8 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:20.928611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd367474-ff07-4dff-b40d-969698b470e8 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:20.974274 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02d79e-bc22-4e29-9b98-2b13c2cb5184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:20.974927 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02d79e-bc22-4e29-9b98-2b13c2cb5184 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:20.975207 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02d79e-bc22-4e29-9b98-2b13c2cb5184 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:20.975478 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02d79e-bc22-4e29-9b98-2b13c2cb5184 None admin] PATH_INFO: `/v3/projects/85168a036bf94350836ae48489878ed3` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:20.976909 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:20.977879 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a02d79e-bc22-4e29-9b98-2b13c2cb5184 None admin] RBAC: Authorizing `identity:get_project(project_id=85168a036bf94350836ae48489878ed3)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:20.979350 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a02d79e-bc22-4e29-9b98-2b13c2cb5184 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:20.981346 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 86/171] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:20 2022] GET /identity/v3/projects/85168a036bf94350836ae48489878ed3 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:20.981774 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02d79e-bc22-4e29-9b98-2b13c2cb5184 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:20.993892 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c138e38-cec5-498c-b3e3-01586fa3a322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:20.994812 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c138e38-cec5-498c-b3e3-01586fa3a322 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:20.995097 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c138e38-cec5-498c-b3e3-01586fa3a322 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:20.995374 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c138e38-cec5-498c-b3e3-01586fa3a322 None admin] PATH_INFO: `/v3/users/706e785a4bca46e7ab7003c417e51cc1` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:20.996562 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c138e38-cec5-498c-b3e3-01586fa3a322 None admin] RBAC: Authorizing `identity:get_user(user_id=706e785a4bca46e7ab7003c417e51cc1)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:20.998028 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c138e38-cec5-498c-b3e3-01586fa3a322 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:21.000857 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 86/172] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:20 2022] GET /identity/v3/users/706e785a4bca46e7ab7003c417e51cc1 => generated 275 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:21.001615 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c138e38-cec5-498c-b3e3-01586fa3a322 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:21.119152 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02d79e-bc22-4e29-9b98-2b13c2cb5184 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:21.173889 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c138e38-cec5-498c-b3e3-01586fa3a322 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:21.214738 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c6e83a-9c37-42fc-b132-016967e5c5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:21.215407 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6e83a-9c37-42fc-b132-016967e5c5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:21.215674 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d93cd-a48b-47cc-9876-b3c8d7d8cca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:21.215791 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6e83a-9c37-42fc-b132-016967e5c5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:21.215948 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6e83a-9c37-42fc-b132-016967e5c5d2 None admin] PATH_INFO: `/v3/users/706e785a4bca46e7ab7003c417e51cc1` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:21.216357 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d93cd-a48b-47cc-9876-b3c8d7d8cca0 None admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:21.216751 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d93cd-a48b-47cc-9876-b3c8d7d8cca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:21.217034 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d93cd-a48b-47cc-9876-b3c8d7d8cca0 None admin] PATH_INFO: `/v3/projects/85168a036bf94350836ae48489878ed3/users/853f632e24be4c118a3ee00e5a2da179/roles/84fc927928374b91847a13b7fbb8ddd1` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:21.217209 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c6e83a-9c37-42fc-b132-016967e5c5d2 None admin] RBAC: Authorizing `identity:get_user(user_id=706e785a4bca46e7ab7003c417e51cc1)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:21.218657 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c6e83a-9c37-42fc-b132-016967e5c5d2 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:21.219038 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d93cd-a48b-47cc-9876-b3c8d7d8cca0 None admin] RBAC: Authorizing `identity:create_grant(project_id=85168a036bf94350836ae48489878ed3, user_id=853f632e24be4c118a3ee00e5a2da179, role_id=84fc927928374b91847a13b7fbb8ddd1)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:21.220274 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 87/173] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:20 2022] GET /identity/v3/users/706e785a4bca46e7ab7003c417e51cc1 => generated 275 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:21.220766 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d93cd-a48b-47cc-9876-b3c8d7d8cca0 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:21.221023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c6e83a-9c37-42fc-b132-016967e5c5d2 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:21.245353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 87/174] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 27 04:08:21 2022] PUT /identity/v3/projects/85168a036bf94350836ae48489878ed3/users/853f632e24be4c118a3ee00e5a2da179/roles/84fc927928374b91847a13b7fbb8ddd1 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:08:21.246251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d93cd-a48b-47cc-9876-b3c8d7d8cca0 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:21.248395 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:21.321430 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:21.382596 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d93cd-a48b-47cc-9876-b3c8d7d8cca0 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:21.449141 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46329b-64f2-4316-a967-1d8e00c1537b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:21.450607 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46329b-64f2-4316-a967-1d8e00c1537b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:21.451196 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46329b-64f2-4316-a967-1d8e00c1537b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:21.452422 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46329b-64f2-4316-a967-1d8e00c1537b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:21.497968 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c6e83a-9c37-42fc-b132-016967e5c5d2 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:21.544442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed46329b-64f2-4316-a967-1d8e00c1537b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:21.547343 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed46329b-64f2-4316-a967-1d8e00c1537b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:21.582683 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d96ee04-af23-41bb-993a-63e36582bb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:21.583663 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96ee04-af23-41bb-993a-63e36582bb94 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:21.584139 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96ee04-af23-41bb-993a-63e36582bb94 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:21.584607 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96ee04-af23-41bb-993a-63e36582bb94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:21.606345 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:21.660505 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 88/175] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:08:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:08:21.665905 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46329b-64f2-4316-a967-1d8e00c1537b None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:21.770500 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d96ee04-af23-41bb-993a-63e36582bb94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:21.771910 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d96ee04-af23-41bb-993a-63e36582bb94 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:21.774783 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46329b-64f2-4316-a967-1d8e00c1537b None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:21.794841 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:21.810403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255d81b-b22b-46ea-b845-5e59b6932618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:21.811064 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255d81b-b22b-46ea-b845-5e59b6932618 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:21.811347 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255d81b-b22b-46ea-b845-5e59b6932618 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:21.811804 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255d81b-b22b-46ea-b845-5e59b6932618 None admin] PATH_INFO: `/v3/projects/85168a036bf94350836ae48489878ed3` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:21.813270 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a255d81b-b22b-46ea-b845-5e59b6932618 None admin] RBAC: Authorizing `identity:get_project(project_id=85168a036bf94350836ae48489878ed3)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:21.815287 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a255d81b-b22b-46ea-b845-5e59b6932618 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:21.817792 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 89/176] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:21 2022] GET /identity/v3/projects/85168a036bf94350836ae48489878ed3 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:08:21.823567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255d81b-b22b-46ea-b845-5e59b6932618 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:21.846695 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:21.941723 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255d81b-b22b-46ea-b845-5e59b6932618 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:21.980051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7002b-3527-4765-8bbc-290fe7f62988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:21.980693 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7002b-3527-4765-8bbc-290fe7f62988 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:21.980978 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7002b-3527-4765-8bbc-290fe7f62988 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:21.981259 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7002b-3527-4765-8bbc-290fe7f62988 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:21.982100 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b7002b-3527-4765-8bbc-290fe7f62988 None admin] RBAC: Adding query filter params (role.id=cdfc87279ea24b65bae3961035133725, scope.project.id=85168a036bf94350836ae48489878ed3, user.id=706e785a4bca46e7ab7003c417e51cc1) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:21.982562 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b7002b-3527-4765-8bbc-290fe7f62988 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:21.984201 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b7002b-3527-4765-8bbc-290fe7f62988 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:21.994711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 90/177] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:08:21 2022] GET /identity/v3/role_assignments?user.id=706e785a4bca46e7ab7003c417e51cc1&scope.project.id=85168a036bf94350836ae48489878ed3&role.id=cdfc87279ea24b65bae3961035133725 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:08:22.045723 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 88/178] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:08:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 825 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:08:22.052232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7002b-3527-4765-8bbc-290fe7f62988 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:22.193357 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7002b-3527-4765-8bbc-290fe7f62988 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:22.196488 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:22.231452 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d845b55-1676-441c-ab97-541e4361439a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:22.232114 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d845b55-1676-441c-ab97-541e4361439a None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:22.232456 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d845b55-1676-441c-ab97-541e4361439a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:22.232740 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d845b55-1676-441c-ab97-541e4361439a None admin] PATH_INFO: `/v3/projects/e6014d57b86c4cf1848ccb82dc88cea9` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:22.234051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d845b55-1676-441c-ab97-541e4361439a None admin] RBAC: Authorizing `identity:get_project(project_id=e6014d57b86c4cf1848ccb82dc88cea9)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:22.235518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d845b55-1676-441c-ab97-541e4361439a None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:22.238215 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 91/179] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:22 2022] GET /identity/v3/projects/e6014d57b86c4cf1848ccb82dc88cea9 => generated 312 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:22.243625 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d96ee04-af23-41bb-993a-63e36582bb94 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:22.375160 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d96ee04-af23-41bb-993a-63e36582bb94 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:22.386247 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:22.418796 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96228146-8fda-4c24-8998-5029e6f7912d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:22.419425 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96228146-8fda-4c24-8998-5029e6f7912d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:22.419706 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96228146-8fda-4c24-8998-5029e6f7912d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:22.419979 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96228146-8fda-4c24-8998-5029e6f7912d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:22.420775 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96228146-8fda-4c24-8998-5029e6f7912d None admin] RBAC: Adding query filter params (role.id=7b6f2ade82434ee79175385de17049eb, scope.project.id=e6014d57b86c4cf1848ccb82dc88cea9, user.id=706e785a4bca46e7ab7003c417e51cc1) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:22.421084 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96228146-8fda-4c24-8998-5029e6f7912d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:22.422570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96228146-8fda-4c24-8998-5029e6f7912d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:22.434447 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 89/180] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:08:22 2022] GET /identity/v3/role_assignments?user.id=706e785a4bca46e7ab7003c417e51cc1&scope.project.id=e6014d57b86c4cf1848ccb82dc88cea9&role.id=7b6f2ade82434ee79175385de17049eb => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:22.561370 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75274473-4ec9-46cd-9663-d07c0a7ad8e7 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:22.561880 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75274473-4ec9-46cd-9663-d07c0a7ad8e7 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:22.561955 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75274473-4ec9-46cd-9663-d07c0a7ad8e7 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:22.563639 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 92/181] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:22.571246 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96ecbc-cd1b-4463-9e45-f15f28c978e3 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:22.571527 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96ecbc-cd1b-4463-9e45-f15f28c978e3 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:22.571874 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96ecbc-cd1b-4463-9e45-f15f28c978e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:22.592502 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-4e96ecbc-cd1b-4463-9e45-f15f28c978e3 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:22.593756 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-4e96ecbc-cd1b-4463-9e45-f15f28c978e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:22.638899 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6436b-1207-4ede-99be-75f35abea5cb None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:22.639188 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6436b-1207-4ede-99be-75f35abea5cb None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:22.639367 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6436b-1207-4ede-99be-75f35abea5cb None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:22.640729 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 93/182] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:22.647071 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c076f-de1c-44cf-9ba5-db2d72a063e7 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:22.647373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c076f-de1c-44cf-9ba5-db2d72a063e7 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:22.647635 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c076f-de1c-44cf-9ba5-db2d72a063e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:22.681243 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-165c076f-de1c-44cf-9ba5-db2d72a063e7 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:22.682902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 90/183] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:22.684476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-165c076f-de1c-44cf-9ba5-db2d72a063e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:22.706933 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e96ecbc-cd1b-4463-9e45-f15f28c978e3 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:22.780227 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 94/184] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 04:08:22.803526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165c076f-de1c-44cf-9ba5-db2d72a063e7 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:22.852397 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e96ecbc-cd1b-4463-9e45-f15f28c978e3 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:22.854535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:22.901738 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:22.909812 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861abab0-fbd8-4c30-8470-877937e96d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:22.910512 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861abab0-fbd8-4c30-8470-877937e96d9d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:22.910795 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861abab0-fbd8-4c30-8470-877937e96d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:22.911069 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861abab0-fbd8-4c30-8470-877937e96d9d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:22.932187 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861abab0-fbd8-4c30-8470-877937e96d9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:22.932393 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165c076f-de1c-44cf-9ba5-db2d72a063e7 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:22.939967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-861abab0-fbd8-4c30-8470-877937e96d9d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 04:08:22.939967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:22.939967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:22.939967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:22.939967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:22.939967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:22.939967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:22.939967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:22.939967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:22.939967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:22.939967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:22.939967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:22.939967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:22.939967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:22.939967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:22.939967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:22.940905 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:22.940905 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:22.940905 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:22.940905 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:22.940905 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:22.940905 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:22.940905 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:22.940905 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:22.940905 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:22.940905 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:22.940905 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:22.940905 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:22.940905 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:22.940905 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:22.940905 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:22.940905 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:22.941692 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:22.941692 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:22.941692 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:22.941692 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:22.941692 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:22.941692 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:22.941692 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:22.941692 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:22.941692 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:22.941692 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:22.941692 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 04:08:22.941692 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:08:22.948302 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 91/185] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 27 04:08:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:08:22.949766 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861abab0-fbd8-4c30-8470-877937e96d9d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:22.980672 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c574ea-5332-4d5a-aba3-4314493b2b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:22.981804 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c574ea-5332-4d5a-aba3-4314493b2b28 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:22.982462 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c574ea-5332-4d5a-aba3-4314493b2b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:22.982937 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c574ea-5332-4d5a-aba3-4314493b2b28 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:22.995023 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c574ea-5332-4d5a-aba3-4314493b2b28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:23.000767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-a3c574ea-5332-4d5a-aba3-4314493b2b28 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:08:23.000767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:23.000767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:23.000767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:23.000767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:23.000767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:23.000767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:23.000767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:23.000767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:23.000767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:23.000767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:23.000767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:23.000767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:23.000767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:23.000767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:23.000767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:23.001700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:23.001700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:23.001700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:23.001700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:23.001700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:23.001700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:23.001700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:23.001700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:23.001700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:23.001700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:23.001700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:23.001700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:23.001700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:23.001700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:23.001700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:23.001700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:23.002594 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:23.002594 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:23.002594 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:23.002594 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:23.002594 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:23.002594 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:23.002594 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:23.002594 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:23.002594 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:23.002594 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:23.002594 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:08:23.002594 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:08:23.005039 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 95/186] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:08:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:23.011925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c574ea-5332-4d5a-aba3-4314493b2b28 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:23.065092 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861abab0-fbd8-4c30-8470-877937e96d9d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:23.100603 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7e62d-e99a-4dcf-8326-5b2644fb7bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:23.101083 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7e62d-e99a-4dcf-8326-5b2644fb7bbc None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:23.101370 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7e62d-e99a-4dcf-8326-5b2644fb7bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:23.101644 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7e62d-e99a-4dcf-8326-5b2644fb7bbc None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:23.102462 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b7e62d-e99a-4dcf-8326-5b2644fb7bbc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:23.102773 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b7e62d-e99a-4dcf-8326-5b2644fb7bbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:23.104170 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b7e62d-e99a-4dcf-8326-5b2644fb7bbc None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:23.117989 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 92/187] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:08:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:23.120126 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7e62d-e99a-4dcf-8326-5b2644fb7bbc None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:23.133297 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c574ea-5332-4d5a-aba3-4314493b2b28 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:23.155020 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:23.177775 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d0e64-e0ab-456c-ad3a-b8b9b06c50e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:23.178601 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d0e64-e0ab-456c-ad3a-b8b9b06c50e2 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:23.179005 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d0e64-e0ab-456c-ad3a-b8b9b06c50e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:23.179674 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d0e64-e0ab-456c-ad3a-b8b9b06c50e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:23.180433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573d0e64-e0ab-456c-ad3a-b8b9b06c50e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:23.180732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573d0e64-e0ab-456c-ad3a-b8b9b06c50e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:23.182488 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573d0e64-e0ab-456c-ad3a-b8b9b06c50e2 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:23.192951 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 96/188] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:08:23.194207 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf6b17-8145-4e2b-9134-0ad70b73c54a None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:23.194494 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf6b17-8145-4e2b-9134-0ad70b73c54a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:23.194764 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf6b17-8145-4e2b-9134-0ad70b73c54a None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:23.196116 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 97/189] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:23.199939 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf6b17-8145-4e2b-9134-0ad70b73c54a None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:23.204149 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:23.252540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7e62d-e99a-4dcf-8326-5b2644fb7bbc None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:23.288846 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84591ca3-3b24-4940-8902-2c55e53daa1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:23.289492 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84591ca3-3b24-4940-8902-2c55e53daa1d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:23.289778 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84591ca3-3b24-4940-8902-2c55e53daa1d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:23.290089 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84591ca3-3b24-4940-8902-2c55e53daa1d None admin] PATH_INFO: `/v3/users/1130e205d84d42a796a43c0153f35b0e` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:23.307732 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84591ca3-3b24-4940-8902-2c55e53daa1d None admin] RBAC: Authorizing `identity:get_user(user_id=1130e205d84d42a796a43c0153f35b0e)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:23.309963 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84591ca3-3b24-4940-8902-2c55e53daa1d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:23.311926 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 93/190] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:23 2022] GET /identity/v3/users/1130e205d84d42a796a43c0153f35b0e => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:08:23.312736 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf6b17-8145-4e2b-9134-0ad70b73c54a None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:23.313230 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b123fb7-16eb-42f5-ae55-c3cc5f6534d8 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:23.313493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b123fb7-16eb-42f5-ae55-c3cc5f6534d8 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:23.313767 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b123fb7-16eb-42f5-ae55-c3cc5f6534d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:23.334742 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-2b123fb7-16eb-42f5-ae55-c3cc5f6534d8 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:23.335923 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2b123fb7-16eb-42f5-ae55-c3cc5f6534d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:23.349018 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1fc119-abbb-49c1-a921-6603c2151580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:23.349707 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1fc119-abbb-49c1-a921-6603c2151580 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:23.350025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1fc119-abbb-49c1-a921-6603c2151580 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:23.350313 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1fc119-abbb-49c1-a921-6603c2151580 None admin] PATH_INFO: `/v3/users/1130e205d84d42a796a43c0153f35b0e` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:23.351549 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1fc119-abbb-49c1-a921-6603c2151580 None admin] RBAC: Authorizing `identity:get_user(user_id=1130e205d84d42a796a43c0153f35b0e)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:23.353011 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1fc119-abbb-49c1-a921-6603c2151580 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:23.358983 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 98/191] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:23 2022] GET /identity/v3/users/1130e205d84d42a796a43c0153f35b0e => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:08:23.358983 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1fc119-abbb-49c1-a921-6603c2151580 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:23.409760 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 94/192] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:23.410872 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b123fb7-16eb-42f5-ae55-c3cc5f6534d8 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:23.440617 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:23.441465 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:23.479595 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1fc119-abbb-49c1-a921-6603c2151580 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:23.515482 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd90f0-5a04-4efa-9b32-9887ccb85e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:23.516115 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd90f0-5a04-4efa-9b32-9887ccb85e51 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:23.516401 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd90f0-5a04-4efa-9b32-9887ccb85e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:23.516684 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd90f0-5a04-4efa-9b32-9887ccb85e51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:23.535065 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b123fb7-16eb-42f5-ae55-c3cc5f6534d8 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:23.580008 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7da831-353e-47db-98ff-8cc4f171b44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:23.581640 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7da831-353e-47db-98ff-8cc4f171b44c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:23.582422 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7da831-353e-47db-98ff-8cc4f171b44c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:23.582897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7da831-353e-47db-98ff-8cc4f171b44c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:23.589177 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dd90f0-5a04-4efa-9b32-9887ccb85e51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:23.590691 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dd90f0-5a04-4efa-9b32-9887ccb85e51 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:23.660566 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7da831-353e-47db-98ff-8cc4f171b44c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:23.662150 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:23.662908 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7da831-353e-47db-98ff-8cc4f171b44c None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:23.679949 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:23.699211 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 99/193] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:08:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:08:23.699211 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd90f0-5a04-4efa-9b32-9887ccb85e51 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:23.784915 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 95/194] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:08:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:08:23.784915 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7da831-353e-47db-98ff-8cc4f171b44c None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:23.801841 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd90f0-5a04-4efa-9b32-9887ccb85e51 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:23.835900 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd2b91-d33c-4cbc-bce5-2084ff3d0d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:23.836544 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd2b91-d33c-4cbc-bce5-2084ff3d0d84 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:23.836828 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd2b91-d33c-4cbc-bce5-2084ff3d0d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:23.837105 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd2b91-d33c-4cbc-bce5-2084ff3d0d84 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:23.838284 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd2b91-d33c-4cbc-bce5-2084ff3d0d84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:23.839733 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd2b91-d33c-4cbc-bce5-2084ff3d0d84 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:23.841314 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 100/195] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:08:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:23.842107 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd2b91-d33c-4cbc-bce5-2084ff3d0d84 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:23.886698 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:23.889546 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7da831-353e-47db-98ff-8cc4f171b44c None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:23.906960 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:23.927982 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aaf166-6660-4768-abd1-1c0cad34d932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:23.928553 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aaf166-6660-4768-abd1-1c0cad34d932 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:23.928833 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aaf166-6660-4768-abd1-1c0cad34d932 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:23.929102 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aaf166-6660-4768-abd1-1c0cad34d932 None admin] PATH_INFO: `/v3/projects/932a11c28fa2405793ce99d7097c63a5` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:23.930274 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aaf166-6660-4768-abd1-1c0cad34d932 None admin] RBAC: Authorizing `identity:get_project(project_id=932a11c28fa2405793ce99d7097c63a5)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:23.931719 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aaf166-6660-4768-abd1-1c0cad34d932 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:23.933537 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 96/196] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:23 2022] GET /identity/v3/projects/932a11c28fa2405793ce99d7097c63a5 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:23.934526 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc030030-71d1-40fe-b7f4-1145a0db7946 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:23.934814 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc030030-71d1-40fe-b7f4-1145a0db7946 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:23.942003 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc030030-71d1-40fe-b7f4-1145a0db7946 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:23.943321 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 97/197] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:23.944090 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc030030-71d1-40fe-b7f4-1145a0db7946 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:23.954073 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd2b91-d33c-4cbc-bce5-2084ff3d0d84 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:23.988402 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3597a-f418-4df5-9c0c-5b59e14bfc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:23.989047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3597a-f418-4df5-9c0c-5b59e14bfc87 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:23.989331 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3597a-f418-4df5-9c0c-5b59e14bfc87 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:23.989607 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3597a-f418-4df5-9c0c-5b59e14bfc87 None admin] PATH_INFO: `/v3/projects/932a11c28fa2405793ce99d7097c63a5` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:23.990823 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3597a-f418-4df5-9c0c-5b59e14bfc87 None admin] RBAC: Authorizing `identity:get_project(project_id=932a11c28fa2405793ce99d7097c63a5)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:23.992259 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3597a-f418-4df5-9c0c-5b59e14bfc87 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:23.993921 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 101/198] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:23 2022] GET /identity/v3/projects/932a11c28fa2405793ce99d7097c63a5 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:23.994705 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3597a-f418-4df5-9c0c-5b59e14bfc87 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:24.047222 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc030030-71d1-40fe-b7f4-1145a0db7946 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:24.081599 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e44eb-b56a-4f6c-9186-4fc7d7117eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:24.082237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e44eb-b56a-4f6c-9186-4fc7d7117eb8 None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:24.082514 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e44eb-b56a-4f6c-9186-4fc7d7117eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:24.082784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e44eb-b56a-4f6c-9186-4fc7d7117eb8 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:24.083469 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066e44eb-b56a-4f6c-9186-4fc7d7117eb8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:24.084917 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066e44eb-b56a-4f6c-9186-4fc7d7117eb8 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:24.093493 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3597a-f418-4df5-9c0c-5b59e14bfc87 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:24.103781 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:24.106000 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 98/199] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 27 04:08:23 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:08:24.107228 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c027ab-873a-4e8c-9f35-2ce153732917 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:24.107505 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c027ab-873a-4e8c-9f35-2ce153732917 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:24.107784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c027ab-873a-4e8c-9f35-2ce153732917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:24.126689 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-66c027ab-873a-4e8c-9f35-2ce153732917 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:24.128530 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-66c027ab-873a-4e8c-9f35-2ce153732917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:24.141288 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9941a-2947-438c-a844-e40c655bc529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:24.142579 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9941a-2947-438c-a844-e40c655bc529 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:24.143768 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9941a-2947-438c-a844-e40c655bc529 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:24.144280 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9941a-2947-438c-a844-e40c655bc529 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:24.146140 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a9941a-2947-438c-a844-e40c655bc529 None admin] RBAC: Adding query filter params (role.id=cdfc87279ea24b65bae3961035133725, scope.project.id=932a11c28fa2405793ce99d7097c63a5, user.id=1130e205d84d42a796a43c0153f35b0e) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:24.146650 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a9941a-2947-438c-a844-e40c655bc529 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:24.148824 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a9941a-2947-438c-a844-e40c655bc529 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:24.158370 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 102/200] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:08:23 2022] GET /identity/v3/role_assignments?user.id=1130e205d84d42a796a43c0153f35b0e&scope.project.id=932a11c28fa2405793ce99d7097c63a5&role.id=cdfc87279ea24b65bae3961035133725 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:24.159900 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9941a-2947-438c-a844-e40c655bc529 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:24.163411 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:24.225421 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 99/201] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:24.254303 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c027ab-873a-4e8c-9f35-2ce153732917 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:24.308039 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9941a-2947-438c-a844-e40c655bc529 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:24.358968 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010ce19-5a9e-4a1d-931e-1eab97e23494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:24.359595 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010ce19-5a9e-4a1d-931e-1eab97e23494 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:24.359874 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010ce19-5a9e-4a1d-931e-1eab97e23494 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:24.360145 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010ce19-5a9e-4a1d-931e-1eab97e23494 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:24.360951 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8010ce19-5a9e-4a1d-931e-1eab97e23494 None admin] RBAC: Adding query filter params (role.id=cf771a8e94ee46149eb081347fea6d67, scope.project.id=932a11c28fa2405793ce99d7097c63a5, user.id=1130e205d84d42a796a43c0153f35b0e) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:24.361258 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8010ce19-5a9e-4a1d-931e-1eab97e23494 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:24.362788 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8010ce19-5a9e-4a1d-931e-1eab97e23494 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:24.368658 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 103/202] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:08:24 2022] GET /identity/v3/role_assignments?user.id=1130e205d84d42a796a43c0153f35b0e&scope.project.id=932a11c28fa2405793ce99d7097c63a5&role.id=cf771a8e94ee46149eb081347fea6d67 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:24.383234 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c027ab-873a-4e8c-9f35-2ce153732917 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:24.429686 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f121af-ffd3-4032-8b7c-87f7037f463c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:24.430402 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f121af-ffd3-4032-8b7c-87f7037f463c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:24.430682 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f121af-ffd3-4032-8b7c-87f7037f463c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:24.430953 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f121af-ffd3-4032-8b7c-87f7037f463c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:24.444451 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f121af-ffd3-4032-8b7c-87f7037f463c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:24.449562 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-01f121af-ffd3-4032-8b7c-87f7037f463c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 04:08:24.449562 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:24.449562 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:24.449562 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:24.449562 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:24.449562 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:24.449562 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:24.449562 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:24.449562 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:24.449562 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:24.449562 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:24.449562 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:24.449562 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:24.449562 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:24.449562 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:24.449562 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:24.450575 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:24.450575 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:24.450575 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:24.450575 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:24.450575 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:24.450575 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:24.450575 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:24.450575 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:24.450575 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:24.450575 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:24.450575 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:24.450575 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:24.450575 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:24.450575 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:24.450575 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:24.450575 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:24.451660 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:24.451660 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:24.451660 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:24.451660 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:24.451660 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:24.451660 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:24.451660 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:24.451660 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:24.451660 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:24.451660 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:24.451660 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 04:08:24.451660 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:08:24.451660 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 100/203] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:24.462039 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010ce19-5a9e-4a1d-931e-1eab97e23494 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:24.505777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:24.584806 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010ce19-5a9e-4a1d-931e-1eab97e23494 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:24.622960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a294f-8e95-47cc-8a29-038cb561fa44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:24.624321 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a294f-8e95-47cc-8a29-038cb561fa44 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:24.624782 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a294f-8e95-47cc-8a29-038cb561fa44 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:24.625232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a294f-8e95-47cc-8a29-038cb561fa44 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:24.626163 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003a294f-8e95-47cc-8a29-038cb561fa44 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:24.626676 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003a294f-8e95-47cc-8a29-038cb561fa44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:24.629004 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003a294f-8e95-47cc-8a29-038cb561fa44 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:24.638503 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 104/204] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 27 04:08:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:08:24.647110 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f121af-ffd3-4032-8b7c-87f7037f463c None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:24.679977 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:24.771806 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f121af-ffd3-4032-8b7c-87f7037f463c None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:24.808470 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc5e3f0-bd66-43a1-85de-31ce1d1fddc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:24.809371 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5e3f0-bd66-43a1-85de-31ce1d1fddc1 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:24.809821 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5e3f0-bd66-43a1-85de-31ce1d1fddc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:24.810266 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5e3f0-bd66-43a1-85de-31ce1d1fddc1 None admin] PATH_INFO: `/v3/users/853f632e24be4c118a3ee00e5a2da179` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:24.811623 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc5e3f0-bd66-43a1-85de-31ce1d1fddc1 None admin] RBAC: Authorizing `identity:get_user(user_id=853f632e24be4c118a3ee00e5a2da179)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:24.813498 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc5e3f0-bd66-43a1-85de-31ce1d1fddc1 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:24.815839 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 101/205] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:24 2022] GET /identity/v3/users/853f632e24be4c118a3ee00e5a2da179 => generated 289 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:24.823071 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a294f-8e95-47cc-8a29-038cb561fa44 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:24.978335 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:24.986271 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a294f-8e95-47cc-8a29-038cb561fa44 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:25.033705 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9030dc-61ec-49ca-8eee-b74adb38a02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:25.034290 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9030dc-61ec-49ca-8eee-b74adb38a02a None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:25.034578 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9030dc-61ec-49ca-8eee-b74adb38a02a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:25.034857 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9030dc-61ec-49ca-8eee-b74adb38a02a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:25.116646 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9030dc-61ec-49ca-8eee-b74adb38a02a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:25.118222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9030dc-61ec-49ca-8eee-b74adb38a02a None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:25.222618 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 105/206] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:08:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:08:25.223273 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:25.230007 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc5e3f0-bd66-43a1-85de-31ce1d1fddc1 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:25.380701 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:25.387584 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc5e3f0-bd66-43a1-85de-31ce1d1fddc1 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:25.435639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5319b5a4-a8fb-4ed4-a104-3a128bb7e232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:25.436288 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5319b5a4-a8fb-4ed4-a104-3a128bb7e232 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:25.436566 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5319b5a4-a8fb-4ed4-a104-3a128bb7e232 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:25.436836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5319b5a4-a8fb-4ed4-a104-3a128bb7e232 None admin] PATH_INFO: `/v3/projects/85168a036bf94350836ae48489878ed3` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:25.442666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5319b5a4-a8fb-4ed4-a104-3a128bb7e232 None admin] RBAC: Authorizing `identity:get_project(project_id=85168a036bf94350836ae48489878ed3)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:25.444108 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5319b5a4-a8fb-4ed4-a104-3a128bb7e232 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:25.446096 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 102/207] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:25 2022] GET /identity/v3/projects/85168a036bf94350836ae48489878ed3 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:25.456740 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9030dc-61ec-49ca-8eee-b74adb38a02a None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:25.598196 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9030dc-61ec-49ca-8eee-b74adb38a02a None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:25.634104 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977c941-af9d-4e71-b28b-a3308e0deb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:25.635489 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977c941-af9d-4e71-b28b-a3308e0deb4e None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:25.635997 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977c941-af9d-4e71-b28b-a3308e0deb4e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:25.636527 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977c941-af9d-4e71-b28b-a3308e0deb4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:25.637581 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f977c941-af9d-4e71-b28b-a3308e0deb4e None admin] RBAC: Adding query filter params (role.id=84fc927928374b91847a13b7fbb8ddd1, scope.project.id=85168a036bf94350836ae48489878ed3, user.id=853f632e24be4c118a3ee00e5a2da179) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:25.638138 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f977c941-af9d-4e71-b28b-a3308e0deb4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:25.639792 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f977c941-af9d-4e71-b28b-a3308e0deb4e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:25.647690 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 106/208] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:08:25 2022] GET /identity/v3/role_assignments?user.id=853f632e24be4c118a3ee00e5a2da179&scope.project.id=85168a036bf94350836ae48489878ed3&role.id=84fc927928374b91847a13b7fbb8ddd1 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:26.607860 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef5d91-f56f-446e-b645-48f54734cc88 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:26.608889 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef5d91-f56f-446e-b645-48f54734cc88 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:26.609368 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef5d91-f56f-446e-b645-48f54734cc88 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:26.610964 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 103/209] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:26.617091 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe3490-f217-43d5-9362-01cdad6a7ae4 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:26.617619 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe3490-f217-43d5-9362-01cdad6a7ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:26.618146 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe3490-f217-43d5-9362-01cdad6a7ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:26.626109 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33893c8-c7c8-4de3-871d-6effd5a6cf2f None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:26.626652 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33893c8-c7c8-4de3-871d-6effd5a6cf2f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:26.627293 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33893c8-c7c8-4de3-871d-6effd5a6cf2f None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:26.628919 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 104/210] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 04:08:26.635183 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635c354-24c9-4e9c-9471-3c17cbdbef4b None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:26.635695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635c354-24c9-4e9c-9471-3c17cbdbef4b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:26.636163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635c354-24c9-4e9c-9471-3c17cbdbef4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:26.638540 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-30fe3490-f217-43d5-9362-01cdad6a7ae4 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:26.639852 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-30fe3490-f217-43d5-9362-01cdad6a7ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:26.656642 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-9635c354-24c9-4e9c-9471-3c17cbdbef4b None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:26.658295 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9635c354-24c9-4e9c-9471-3c17cbdbef4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:26.708282 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:26.720259 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 107/211] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:26.727325 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777cbbb-94d7-43cd-b66c-507c9668d3ae None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:26.727611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777cbbb-94d7-43cd-b66c-507c9668d3ae None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:26.727889 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777cbbb-94d7-43cd-b66c-507c9668d3ae None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:26.729217 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 108/212] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:26 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:26.734643 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bacee-c945-45b0-b449-be543e2eae31 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:26.734932 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bacee-c945-45b0-b449-be543e2eae31 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:26.735206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bacee-c945-45b0-b449-be543e2eae31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:26.742490 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 105/213] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 04:08:26.744400 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635c354-24c9-4e9c-9471-3c17cbdbef4b None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:26.754211 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-dd9bacee-c945-45b0-b449-be543e2eae31 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:26.755646 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-dd9bacee-c945-45b0-b449-be543e2eae31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:26.824426 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 109/214] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:26.830187 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bacee-c945-45b0-b449-be543e2eae31 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:26.852126 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:26.862884 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635c354-24c9-4e9c-9471-3c17cbdbef4b None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:26.897514 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2265855c-81aa-4e11-88d1-93115acf7d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:26.898201 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265855c-81aa-4e11-88d1-93115acf7d6c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:26.898489 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265855c-81aa-4e11-88d1-93115acf7d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:26.898763 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265855c-81aa-4e11-88d1-93115acf7d6c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:26.909349 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2265855c-81aa-4e11-88d1-93115acf7d6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:26.914302 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-2265855c-81aa-4e11-88d1-93115acf7d6c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:08:26.914302 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:26.914302 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:26.914302 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:26.914302 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:26.914302 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:26.914302 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:26.914302 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:26.914302 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:26.914302 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:26.914302 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:26.914302 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:26.914302 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:26.914302 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:26.914302 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:26.914302 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:26.915226 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:26.915226 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:26.915226 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:26.915226 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:26.915226 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:26.915226 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:26.915226 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:26.915226 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:26.915226 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:26.915226 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:26.915226 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:26.915226 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:26.915226 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:26.915226 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:26.915226 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:26.915226 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:26.916374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:26.916374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:26.916374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:26.916374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:26.916374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:26.916374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:26.916374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:26.916374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:26.916374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:26.916374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:26.916374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:08:26.916374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:08:26.916374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 106/215] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:08:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:26.916374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2265855c-81aa-4e11-88d1-93115acf7d6c None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:26.933626 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bacee-c945-45b0-b449-be543e2eae31 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:26.956836 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:26.968803 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0525581-d656-4f31-8a00-91fc0c81d86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:26.969392 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0525581-d656-4f31-8a00-91fc0c81d86f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:26.969673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0525581-d656-4f31-8a00-91fc0c81d86f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:26.969949 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0525581-d656-4f31-8a00-91fc0c81d86f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:26.980483 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0525581-d656-4f31-8a00-91fc0c81d86f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:26.985399 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-e0525581-d656-4f31-8a00-91fc0c81d86f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:26.985399 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:26.985399 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:26.985399 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:26.985399 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:26.985399 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:26.985399 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:26.985399 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:26.985399 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:26.985399 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:26.985399 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:26.985399 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:26.985399 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:26.985399 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:26.985399 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:26.985399 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:26.986382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:26.986382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:26.986382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:26.986382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:26.986382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:26.986382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:26.986382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:26.986382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:26.986382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:26.986382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:26.986382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:26.986382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:26.986382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:26.986382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:26.986382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:26.986382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:26.987349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:26.987349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:26.987349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:26.987349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:26.987349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:26.987349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:26.987349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:26.987349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:26.987349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:26.987349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:26.987349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:26.987349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:08:26.987349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 110/216] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:26.988004 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1d36d-e3ae-4501-b66f-961b64302eea None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:26.988070 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1d36d-e3ae-4501-b66f-961b64302eea None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:26.988987 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1d36d-e3ae-4501-b66f-961b64302eea None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:26.989743 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 111/217] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:26.994229 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e1d36d-e3ae-4501-b66f-961b64302eea None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:27.017598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2265855c-81aa-4e11-88d1-93115acf7d6c None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:27.051404 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8781e-f24c-4a43-8640-1974bb163c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:27.052049 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8781e-f24c-4a43-8640-1974bb163c8f None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:27.052334 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8781e-f24c-4a43-8640-1974bb163c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:27.052610 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8781e-f24c-4a43-8640-1974bb163c8f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:27.063364 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba8781e-f24c-4a43-8640-1974bb163c8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:27.068963 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-0ba8781e-f24c-4a43-8640-1974bb163c8f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 04:08:27.068963 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:27.068963 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:27.068963 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:27.068963 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:27.068963 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:27.068963 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:27.068963 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:27.068963 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:27.068963 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:27.068963 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:27.068963 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:27.068963 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:27.068963 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:27.068963 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:27.068963 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:27.069860 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:27.069860 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:27.069860 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:27.069860 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:27.069860 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:27.069860 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:27.069860 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:27.069860 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:27.069860 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:27.069860 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:27.069860 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:27.069860 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:27.069860 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:27.069860 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:27.069860 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:27.069860 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:27.071030 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:27.071030 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:27.071030 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:27.071030 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:27.071030 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:27.071030 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:27.071030 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:27.071030 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:27.071030 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:27.071030 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:27.071030 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 04:08:27.071030 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:08:27.071030 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 107/218] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 27 04:08:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:27.071030 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8781e-f24c-4a43-8640-1974bb163c8f None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:27.086173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:27.099744 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e1d36d-e3ae-4501-b66f-961b64302eea None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:27.132766 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30295b94-dc31-456d-aac1-6129ffd82634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:27.133387 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30295b94-dc31-456d-aac1-6129ffd82634 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:27.133664 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30295b94-dc31-456d-aac1-6129ffd82634 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:27.133940 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30295b94-dc31-456d-aac1-6129ffd82634 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:27.134720 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30295b94-dc31-456d-aac1-6129ffd82634 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:27.135012 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30295b94-dc31-456d-aac1-6129ffd82634 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:27.136373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30295b94-dc31-456d-aac1-6129ffd82634 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:27.144944 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 112/219] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:27.147305 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8922017-ae51-4326-8732-b8b47dc094e9 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:27.147589 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8922017-ae51-4326-8732-b8b47dc094e9 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:27.147860 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8922017-ae51-4326-8732-b8b47dc094e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:27.165835 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-a8922017-ae51-4326-8732-b8b47dc094e9 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:27.167060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a8922017-ae51-4326-8732-b8b47dc094e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:27.173058 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8781e-f24c-4a43-8640-1974bb163c8f None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:27.206614 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b1e11e-c199-4f36-96eb-7430bf8fd8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:27.207173 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:27.207291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1e11e-c199-4f36-96eb-7430bf8fd8dc None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:27.207503 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1e11e-c199-4f36-96eb-7430bf8fd8dc None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:27.207788 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1e11e-c199-4f36-96eb-7430bf8fd8dc None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:27.208677 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b1e11e-c199-4f36-96eb-7430bf8fd8dc None admin] RBAC: Adding query filter params (name=member) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:27.209013 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b1e11e-c199-4f36-96eb-7430bf8fd8dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:27.210471 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b1e11e-c199-4f36-96eb-7430bf8fd8dc None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:27.218708 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 108/220] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 27 04:08:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:27.219362 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b1e11e-c199-4f36-96eb-7430bf8fd8dc None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:27.240685 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 113/221] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:27.241521 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8922017-ae51-4326-8732-b8b47dc094e9 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:27.294137 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:27.321658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b1e11e-c199-4f36-96eb-7430bf8fd8dc None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:27.341347 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8922017-ae51-4326-8732-b8b47dc094e9 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:27.354039 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b0662-5128-47f6-a16b-abe9aeec8c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:27.354660 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b0662-5128-47f6-a16b-abe9aeec8c67 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:27.354942 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b0662-5128-47f6-a16b-abe9aeec8c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:27.355213 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b0662-5128-47f6-a16b-abe9aeec8c67 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:27.355930 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b0662-5128-47f6-a16b-abe9aeec8c67 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:27.356233 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b0662-5128-47f6-a16b-abe9aeec8c67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:27.357611 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b0662-5128-47f6-a16b-abe9aeec8c67 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:27.365728 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 109/222] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:08:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:27.366535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b0662-5128-47f6-a16b-abe9aeec8c67 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:27.379364 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4116d51a-b91e-497b-baf6-338b87c63353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:27.380020 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116d51a-b91e-497b-baf6-338b87c63353 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:27.380304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116d51a-b91e-497b-baf6-338b87c63353 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:27.380579 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116d51a-b91e-497b-baf6-338b87c63353 None admin] PATH_INFO: `/v3/users/1130e205d84d42a796a43c0153f35b0e` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:27.381752 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4116d51a-b91e-497b-baf6-338b87c63353 None admin] RBAC: Authorizing `identity:get_user(user_id=1130e205d84d42a796a43c0153f35b0e)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:27.383227 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4116d51a-b91e-497b-baf6-338b87c63353 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:27.384825 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 114/223] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:27 2022] GET /identity/v3/users/1130e205d84d42a796a43c0153f35b0e => generated 283 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:27.385582 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4116d51a-b91e-497b-baf6-338b87c63353 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:27.428635 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:27.468357 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b0662-5128-47f6-a16b-abe9aeec8c67 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:27.489236 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4116d51a-b91e-497b-baf6-338b87c63353 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:27.501410 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66635a35-6a60-4ba7-bed4-15cdf2730e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:27.502085 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66635a35-6a60-4ba7-bed4-15cdf2730e21 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:27.502388 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66635a35-6a60-4ba7-bed4-15cdf2730e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:27.502657 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66635a35-6a60-4ba7-bed4-15cdf2730e21 None admin] PATH_INFO: `/v3/users/933e108d6dff4162a70ed6be5436900b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:27.505420 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:27.516071 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66635a35-6a60-4ba7-bed4-15cdf2730e21 None admin] RBAC: Authorizing `identity:get_user(user_id=933e108d6dff4162a70ed6be5436900b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:27.517518 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66635a35-6a60-4ba7-bed4-15cdf2730e21 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:27.519823 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 110/224] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:27 2022] GET /identity/v3/users/933e108d6dff4162a70ed6be5436900b => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:27.520582 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66635a35-6a60-4ba7-bed4-15cdf2730e21 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:27.522826 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c060f63-3022-4f61-aa51-0ba18f994d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:27.523475 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c060f63-3022-4f61-aa51-0ba18f994d26 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:27.523754 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c060f63-3022-4f61-aa51-0ba18f994d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:27.524040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c060f63-3022-4f61-aa51-0ba18f994d26 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:27.525492 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c060f63-3022-4f61-aa51-0ba18f994d26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:27.526954 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c060f63-3022-4f61-aa51-0ba18f994d26 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:27.528531 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 115/225] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:08:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:27.529273 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c060f63-3022-4f61-aa51-0ba18f994d26 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:27.622466 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66635a35-6a60-4ba7-bed4-15cdf2730e21 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:27.629285 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c060f63-3022-4f61-aa51-0ba18f994d26 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:27.640122 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:27.657290 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8624b59a-c839-41ae-8fc4-05c6b3f141fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:27.657928 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624b59a-c839-41ae-8fc4-05c6b3f141fb None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:27.658234 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624b59a-c839-41ae-8fc4-05c6b3f141fb None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:27.659040 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624b59a-c839-41ae-8fc4-05c6b3f141fb None admin] PATH_INFO: `/v3/users/1130e205d84d42a796a43c0153f35b0e` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:27.660218 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8624b59a-c839-41ae-8fc4-05c6b3f141fb None admin] RBAC: Authorizing `identity:get_user(user_id=1130e205d84d42a796a43c0153f35b0e)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:27.661782 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8624b59a-c839-41ae-8fc4-05c6b3f141fb None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:27.663411 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 111/226] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:27 2022] GET /identity/v3/users/1130e205d84d42a796a43c0153f35b0e => generated 283 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:27.664169 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8624b59a-c839-41ae-8fc4-05c6b3f141fb None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:27.664357 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a182f6c4-9099-4bd2-b0e3-0aae2247d7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:27.665000 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a182f6c4-9099-4bd2-b0e3-0aae2247d7c1 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:27.665278 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a182f6c4-9099-4bd2-b0e3-0aae2247d7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:27.665551 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a182f6c4-9099-4bd2-b0e3-0aae2247d7c1 None admin] PATH_INFO: `/v3/projects/932a11c28fa2405793ce99d7097c63a5` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:27.666724 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a182f6c4-9099-4bd2-b0e3-0aae2247d7c1 None admin] RBAC: Authorizing `identity:get_project(project_id=932a11c28fa2405793ce99d7097c63a5)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:27.668159 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a182f6c4-9099-4bd2-b0e3-0aae2247d7c1 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:27.669805 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 116/227] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:27 2022] GET /identity/v3/projects/932a11c28fa2405793ce99d7097c63a5 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:27.670567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a182f6c4-9099-4bd2-b0e3-0aae2247d7c1 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:27.725819 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:27.763434 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8624b59a-c839-41ae-8fc4-05c6b3f141fb None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:27.771105 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a182f6c4-9099-4bd2-b0e3-0aae2247d7c1 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:27.797369 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cac9d5-ef2a-4fc4-b64f-9fa81ed2694c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:27.798016 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cac9d5-ef2a-4fc4-b64f-9fa81ed2694c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:27.798299 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cac9d5-ef2a-4fc4-b64f-9fa81ed2694c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:27.798570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cac9d5-ef2a-4fc4-b64f-9fa81ed2694c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:27.804247 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f943813e-f1e0-4ada-b710-5e776d7577ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:27.804862 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943813e-f1e0-4ada-b710-5e776d7577ca None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:27.805143 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943813e-f1e0-4ada-b710-5e776d7577ca None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:27.805418 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943813e-f1e0-4ada-b710-5e776d7577ca None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:27.806135 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f943813e-f1e0-4ada-b710-5e776d7577ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:27.807575 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f943813e-f1e0-4ada-b710-5e776d7577ca None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:27.829198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 117/228] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 27 04:08:27 2022] POST /identity/v3/users => generated 297 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:08:27.829198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f943813e-f1e0-4ada-b710-5e776d7577ca None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:27.868968 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:27.885783 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cac9d5-ef2a-4fc4-b64f-9fa81ed2694c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:27.887376 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cac9d5-ef2a-4fc4-b64f-9fa81ed2694c None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:27.949805 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f943813e-f1e0-4ada-b710-5e776d7577ca None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:27.956668 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:27.992313 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3375b8-ff13-442e-84dc-c733454f65e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:27.992979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3375b8-ff13-442e-84dc-c733454f65e8 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:27.993272 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3375b8-ff13-442e-84dc-c733454f65e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:27.993555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3375b8-ff13-442e-84dc-c733454f65e8 None admin] PATH_INFO: `/v3/projects/932a11c28fa2405793ce99d7097c63a5` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:27.994787 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3375b8-ff13-442e-84dc-c733454f65e8 None admin] RBAC: Authorizing `identity:get_project(project_id=932a11c28fa2405793ce99d7097c63a5)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:27.996251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3375b8-ff13-442e-84dc-c733454f65e8 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:27.997928 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 118/229] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:27 2022] GET /identity/v3/projects/932a11c28fa2405793ce99d7097c63a5 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:27.998736 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3375b8-ff13-442e-84dc-c733454f65e8 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:28.004079 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 112/230] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:08:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:08:28.005913 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cac9d5-ef2a-4fc4-b64f-9fa81ed2694c None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:28.116501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3375b8-ff13-442e-84dc-c733454f65e8 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:28.118783 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:28.126422 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cac9d5-ef2a-4fc4-b64f-9fa81ed2694c None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:28.164221 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba5c9a8-61c9-4d92-8c8a-5e681c1b4a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:28.164916 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5c9a8-61c9-4d92-8c8a-5e681c1b4a06 None admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:28.165201 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5c9a8-61c9-4d92-8c8a-5e681c1b4a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:28.165478 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5c9a8-61c9-4d92-8c8a-5e681c1b4a06 None admin] PATH_INFO: `/v3/projects/932a11c28fa2405793ce99d7097c63a5/users/1130e205d84d42a796a43c0153f35b0e/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:28.167560 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba5c9a8-61c9-4d92-8c8a-5e681c1b4a06 None admin] RBAC: Authorizing `identity:create_grant(project_id=932a11c28fa2405793ce99d7097c63a5, user_id=1130e205d84d42a796a43c0153f35b0e, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:28.168953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba5c9a8-61c9-4d92-8c8a-5e681c1b4a06 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:28.173283 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785ad230-6649-4256-83c7-a0728bbc91e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:28.175101 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ad230-6649-4256-83c7-a0728bbc91e6 None admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:28.175101 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ad230-6649-4256-83c7-a0728bbc91e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:28.175270 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ad230-6649-4256-83c7-a0728bbc91e6 None admin] PATH_INFO: `/v3/projects/932a11c28fa2405793ce99d7097c63a5/users/1130e205d84d42a796a43c0153f35b0e/roles/cdfc87279ea24b65bae3961035133725` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:28.177916 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785ad230-6649-4256-83c7-a0728bbc91e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=932a11c28fa2405793ce99d7097c63a5, user_id=1130e205d84d42a796a43c0153f35b0e, role_id=cdfc87279ea24b65bae3961035133725)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:28.179526 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785ad230-6649-4256-83c7-a0728bbc91e6 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:28.187846 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 119/231] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 27 04:08:27 2022] PUT /identity/v3/projects/932a11c28fa2405793ce99d7097c63a5/users/1130e205d84d42a796a43c0153f35b0e/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:08:28.189159 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba5c9a8-61c9-4d92-8c8a-5e681c1b4a06 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:28.203682 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 113/232] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 27 04:08:28 2022] PUT /identity/v3/projects/932a11c28fa2405793ce99d7097c63a5/users/1130e205d84d42a796a43c0153f35b0e/roles/cdfc87279ea24b65bae3961035133725 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:08:28.343747 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba5c9a8-61c9-4d92-8c8a-5e681c1b4a06 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:28.403483 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e37fd9-1d2d-487e-8503-5f949c7e23e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:28.404072 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e37fd9-1d2d-487e-8503-5f949c7e23e4 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:28.404359 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e37fd9-1d2d-487e-8503-5f949c7e23e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:28.404637 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e37fd9-1d2d-487e-8503-5f949c7e23e4 None admin] PATH_INFO: `/v3/projects/932a11c28fa2405793ce99d7097c63a5` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:28.405818 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e37fd9-1d2d-487e-8503-5f949c7e23e4 None admin] RBAC: Authorizing `identity:get_project(project_id=932a11c28fa2405793ce99d7097c63a5)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:28.407308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e37fd9-1d2d-487e-8503-5f949c7e23e4 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:28.409804 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 120/233] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:28 2022] GET /identity/v3/projects/932a11c28fa2405793ce99d7097c63a5 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:28.415862 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785ad230-6649-4256-83c7-a0728bbc91e6 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:28.444919 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:28.545669 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785ad230-6649-4256-83c7-a0728bbc91e6 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:28.583559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab06aa-2541-4b28-b5b7-8a8372227cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:28.584190 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab06aa-2541-4b28-b5b7-8a8372227cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:28.584473 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab06aa-2541-4b28-b5b7-8a8372227cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:28.584746 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab06aa-2541-4b28-b5b7-8a8372227cb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:28.585626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab06aa-2541-4b28-b5b7-8a8372227cb2 None admin] RBAC: Adding query filter params (role.id=7b6f2ade82434ee79175385de17049eb, scope.project.id=932a11c28fa2405793ce99d7097c63a5, user.id=933e108d6dff4162a70ed6be5436900b) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:28.585935 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab06aa-2541-4b28-b5b7-8a8372227cb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:28.587534 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab06aa-2541-4b28-b5b7-8a8372227cb2 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:28.593700 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 114/234] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:08:28 2022] GET /identity/v3/role_assignments?user.id=933e108d6dff4162a70ed6be5436900b&scope.project.id=932a11c28fa2405793ce99d7097c63a5&role.id=7b6f2ade82434ee79175385de17049eb => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:30.388788 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda6875-4928-4b57-8580-f21987b761a4 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:30.389844 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda6875-4928-4b57-8580-f21987b761a4 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:30.390369 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda6875-4928-4b57-8580-f21987b761a4 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:30.392275 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 121/235] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:30.398887 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455f341-aa07-43d3-b581-9b8280cab1c2 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:30.399763 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455f341-aa07-43d3-b581-9b8280cab1c2 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:30.400266 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455f341-aa07-43d3-b581-9b8280cab1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:30.426276 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-d455f341-aa07-43d3-b581-9b8280cab1c2 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:30.427803 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d455f341-aa07-43d3-b581-9b8280cab1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:30.536127 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 115/236] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 04:08:30.547415 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637b87d-6cd1-43cf-ab22-ea8e724dca13 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:30.548003 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637b87d-6cd1-43cf-ab22-ea8e724dca13 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:30.548481 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637b87d-6cd1-43cf-ab22-ea8e724dca13 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:30.551377 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 122/237] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:30.555940 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964ccba-76e0-40e5-b137-49cbdd310c3a None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:30.556491 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964ccba-76e0-40e5-b137-49cbdd310c3a None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:30.556977 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964ccba-76e0-40e5-b137-49cbdd310c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:30.569491 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b637b87d-6cd1-43cf-ab22-ea8e724dca13 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:30.576083 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-2964ccba-76e0-40e5-b137-49cbdd310c3a None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:30.578689 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2964ccba-76e0-40e5-b137-49cbdd310c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:30.591195 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:30.636142 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:30.687725 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 116/238] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:30.727669 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b637b87d-6cd1-43cf-ab22-ea8e724dca13 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:30.735594 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9148932-082e-4131-8cd4-c86fe34e7d4a None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:30.735883 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9148932-082e-4131-8cd4-c86fe34e7d4a None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:30.736165 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9148932-082e-4131-8cd4-c86fe34e7d4a None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:30.737933 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 117/239] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:30.743122 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143e357-0e65-4554-b3bf-ddf23eb7331e None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:30.743671 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143e357-0e65-4554-b3bf-ddf23eb7331e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:30.744152 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143e357-0e65-4554-b3bf-ddf23eb7331e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:30.768860 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f833f956-ab96-4546-8e83-db8ab4928e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:30.769634 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-d143e357-0e65-4554-b3bf-ddf23eb7331e None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:30.770093 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833f956-ab96-4546-8e83-db8ab4928e8f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:30.770528 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833f956-ab96-4546-8e83-db8ab4928e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:30.770953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833f956-ab96-4546-8e83-db8ab4928e8f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:30.771359 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d143e357-0e65-4554-b3bf-ddf23eb7331e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:30.772302 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f833f956-ab96-4546-8e83-db8ab4928e8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:30.773958 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f833f956-ab96-4546-8e83-db8ab4928e8f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:30.776830 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 123/240] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:08:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:30.777636 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f833f956-ab96-4546-8e83-db8ab4928e8f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:30.857074 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 118/241] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 04:08:30.858275 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143e357-0e65-4554-b3bf-ddf23eb7331e None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:30.893115 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:30.897334 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f833f956-ab96-4546-8e83-db8ab4928e8f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:30.934198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd558e-6fc5-4674-8551-397e841e72d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:30.935197 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd558e-6fc5-4674-8551-397e841e72d9 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:30.935703 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd558e-6fc5-4674-8551-397e841e72d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:30.936179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd558e-6fc5-4674-8551-397e841e72d9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:30.947807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fd558e-6fc5-4674-8551-397e841e72d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:30.954572 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-c1fd558e-6fc5-4674-8551-397e841e72d9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 04:08:30.954572 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:30.954572 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:30.954572 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:30.954572 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:30.954572 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:30.954572 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:30.954572 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:30.954572 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:30.954572 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:30.954572 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:30.954572 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:30.954572 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:30.954572 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:30.954572 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:30.954572 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:30.955531 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:30.955531 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:30.955531 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:30.955531 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:30.955531 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:30.955531 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:30.955531 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:30.955531 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:30.955531 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:30.955531 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:30.955531 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:30.955531 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:30.955531 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:30.955531 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:30.955531 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:30.955531 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:30.956476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:30.956476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:30.956476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:30.956476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:30.956476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:30.956476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:30.956476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:30.956476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:30.956476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:30.956476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:30.956476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 04:08:30.956476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:08:30.958448 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:30.959812 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 124/242] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:30.961009 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd558e-6fc5-4674-8551-397e841e72d9 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:30.983049 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143e357-0e65-4554-b3bf-ddf23eb7331e None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:31.020341 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f32137-b8f5-4501-a872-3145ef8e0fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:31.021212 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f32137-b8f5-4501-a872-3145ef8e0fc0 None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:31.021684 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f32137-b8f5-4501-a872-3145ef8e0fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:31.021962 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f32137-b8f5-4501-a872-3145ef8e0fc0 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:31.022687 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f32137-b8f5-4501-a872-3145ef8e0fc0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:31.024136 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f32137-b8f5-4501-a872-3145ef8e0fc0 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:31.053214 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 119/243] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 27 04:08:30 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:08:31.053214 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f32137-b8f5-4501-a872-3145ef8e0fc0 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:31.072160 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd558e-6fc5-4674-8551-397e841e72d9 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:31.115169 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92834c14-edd4-4680-b489-457c075d06eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:31.115964 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92834c14-edd4-4680-b489-457c075d06eb None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:31.116350 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92834c14-edd4-4680-b489-457c075d06eb None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:31.116645 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92834c14-edd4-4680-b489-457c075d06eb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:31.129154 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92834c14-edd4-4680-b489-457c075d06eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:31.135088 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-92834c14-edd4-4680-b489-457c075d06eb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 04:08:31.135088 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:31.135088 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:31.135088 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:31.135088 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:31.135088 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:31.135088 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:31.135088 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:31.135088 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:31.135088 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:31.135088 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:31.135088 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:31.135088 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:31.135088 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:31.135088 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:31.135088 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:31.136060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:31.136060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:31.136060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:31.136060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:31.136060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:31.136060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:31.136060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:31.136060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:31.136060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:31.136060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:31.136060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:31.136060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:31.136060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:31.136060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:31.136060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:31.136060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:31.136835 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:31.136835 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:31.136835 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:31.136835 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:31.136835 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:31.136835 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:31.136835 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:31.136835 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:31.136835 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:31.136835 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:31.136835 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 04:08:31.136835 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:08:31.137924 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 125/244] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 27 04:08:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:31.139007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4f4b7-0ca5-4d02-859b-4dc112810922 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:31.139255 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4f4b7-0ca5-4d02-859b-4dc112810922 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:31.139482 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4f4b7-0ca5-4d02-859b-4dc112810922 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:31.141079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 126/245] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:31.143231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e19073-5f89-4fc8-b360-c34b29be7e1c None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:31.143738 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e19073-5f89-4fc8-b360-c34b29be7e1c None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:31.144227 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e19073-5f89-4fc8-b360-c34b29be7e1c None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:31.146491 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 127/246] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:31.154027 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e19073-5f89-4fc8-b360-c34b29be7e1c None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:31.178549 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f32137-b8f5-4501-a872-3145ef8e0fc0 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:31.179028 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:31.212650 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:31.222952 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87a0ac-2b98-42a3-8732-3afc43f69e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:31.224639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87a0ac-2b98-42a3-8732-3afc43f69e06 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:31.224971 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87a0ac-2b98-42a3-8732-3afc43f69e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:31.225201 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87a0ac-2b98-42a3-8732-3afc43f69e06 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:31.225951 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed87a0ac-2b98-42a3-8732-3afc43f69e06 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:31.227065 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed87a0ac-2b98-42a3-8732-3afc43f69e06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:31.228515 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed87a0ac-2b98-42a3-8732-3afc43f69e06 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:31.241998 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 120/247] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 27 04:08:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:31.241998 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51552d7-9e37-4e0a-b631-a9d61a3b9681 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:31.241998 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51552d7-9e37-4e0a-b631-a9d61a3b9681 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:31.241998 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51552d7-9e37-4e0a-b631-a9d61a3b9681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:31.260793 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-d51552d7-9e37-4e0a-b631-a9d61a3b9681 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:31.262006 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d51552d7-9e37-4e0a-b631-a9d61a3b9681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:31.277609 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e19073-5f89-4fc8-b360-c34b29be7e1c None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:31.315829 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bf4e03-d02e-4647-b5f3-23edc13637d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:31.316467 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf4e03-d02e-4647-b5f3-23edc13637d3 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:31.316744 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf4e03-d02e-4647-b5f3-23edc13637d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:31.317016 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf4e03-d02e-4647-b5f3-23edc13637d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:31.317799 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bf4e03-d02e-4647-b5f3-23edc13637d3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:31.318076 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bf4e03-d02e-4647-b5f3-23edc13637d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:31.319503 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bf4e03-d02e-4647-b5f3-23edc13637d3 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:31.330012 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 128/248] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:08:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:31.330833 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bc190-a397-4ed5-8520-d2fceeb0d46f None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:31.331774 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bc190-a397-4ed5-8520-d2fceeb0d46f None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:31.332411 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bc190-a397-4ed5-8520-d2fceeb0d46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:31.348602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 121/249] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:31.349471 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51552d7-9e37-4e0a-b631-a9d61a3b9681 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:31.363969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-4d7bc190-a397-4ed5-8520-d2fceeb0d46f None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:31.365203 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-4d7bc190-a397-4ed5-8520-d2fceeb0d46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:31.445597 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 129/250] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:31.446496 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7bc190-a397-4ed5-8520-d2fceeb0d46f None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:31.473594 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:31.481421 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51552d7-9e37-4e0a-b631-a9d61a3b9681 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:31.515266 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:31.531471 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a4340e-6839-470e-9a58-b365bb1ba712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:31.532187 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4340e-6839-470e-9a58-b365bb1ba712 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:31.532612 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4340e-6839-470e-9a58-b365bb1ba712 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:31.533059 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4340e-6839-470e-9a58-b365bb1ba712 None admin] PATH_INFO: `/v3/users/ef7b73952eec4ef6b1a0688fce3b4dd4` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:31.554291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a4340e-6839-470e-9a58-b365bb1ba712 None admin] RBAC: Authorizing `identity:get_user(user_id=ef7b73952eec4ef6b1a0688fce3b4dd4)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:31.555776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a4340e-6839-470e-9a58-b365bb1ba712 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:31.558799 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 122/251] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:31 2022] GET /identity/v3/users/ef7b73952eec4ef6b1a0688fce3b4dd4 => generated 297 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:31.562693 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a4340e-6839-470e-9a58-b365bb1ba712 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:31.600835 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7bc190-a397-4ed5-8520-d2fceeb0d46f None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:31.642460 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb677d5-0f2a-4981-b59d-bb781d22b8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:31.644050 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb677d5-0f2a-4981-b59d-bb781d22b8a7 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:31.644540 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb677d5-0f2a-4981-b59d-bb781d22b8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:31.645021 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb677d5-0f2a-4981-b59d-bb781d22b8a7 None admin] PATH_INFO: `/v3/users/1130e205d84d42a796a43c0153f35b0e` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:31.646452 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb677d5-0f2a-4981-b59d-bb781d22b8a7 None admin] RBAC: Authorizing `identity:get_user(user_id=1130e205d84d42a796a43c0153f35b0e)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:31.648111 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb677d5-0f2a-4981-b59d-bb781d22b8a7 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:31.649944 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 130/252] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:31 2022] GET /identity/v3/users/1130e205d84d42a796a43c0153f35b0e => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:31.650970 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb677d5-0f2a-4981-b59d-bb781d22b8a7 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:31.685539 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a4340e-6839-470e-9a58-b365bb1ba712 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:31.731545 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a370ba-6894-4a6b-b04e-6907c5c55147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:31.732327 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a370ba-6894-4a6b-b04e-6907c5c55147 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:31.732456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a370ba-6894-4a6b-b04e-6907c5c55147 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:31.732725 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a370ba-6894-4a6b-b04e-6907c5c55147 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:31.737102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:31.751269 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a370ba-6894-4a6b-b04e-6907c5c55147 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:31.757271 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-c2a370ba-6894-4a6b-b04e-6907c5c55147 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:08:31.757271 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:31.757271 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:31.757271 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:31.757271 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:31.757271 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:31.757271 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:31.757271 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:31.757271 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:31.757271 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:31.757271 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:31.757271 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:31.757271 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:31.757271 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:31.757271 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:31.757271 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:31.758228 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:31.758228 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:31.758228 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:31.758228 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:31.758228 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:31.758228 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:31.758228 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:31.758228 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:31.758228 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:31.758228 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:31.758228 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:31.758228 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:31.758228 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:31.758228 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:31.758228 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:31.758228 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:31.759037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:31.759037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:31.759037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:31.759037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:31.759037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:31.759037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:31.759037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:31.759037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:31.759037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:31.759037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:31.759037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:08:31.759037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:08:31.759037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 123/253] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:08:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:31.759687 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a370ba-6894-4a6b-b04e-6907c5c55147 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:31.776080 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb677d5-0f2a-4981-b59d-bb781d22b8a7 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:31.818124 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:31.833032 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf606f3d-72db-4600-9bd7-2de1b1c39975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:31.833695 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf606f3d-72db-4600-9bd7-2de1b1c39975 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:31.834006 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf606f3d-72db-4600-9bd7-2de1b1c39975 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:31.834276 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf606f3d-72db-4600-9bd7-2de1b1c39975 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:31.852767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf606f3d-72db-4600-9bd7-2de1b1c39975 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:31.859933 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-bf606f3d-72db-4600-9bd7-2de1b1c39975 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:31.859933 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:31.859933 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:31.859933 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:31.859933 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:31.859933 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:31.859933 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:31.859933 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:31.859933 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:31.859933 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:31.859933 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:31.859933 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:31.859933 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:31.859933 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:31.859933 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:31.859933 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:31.860823 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:31.860823 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:31.860823 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:31.860823 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:31.860823 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:31.860823 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:31.860823 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:31.860823 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:31.860823 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:31.860823 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:31.860823 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:31.860823 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:31.860823 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:31.860823 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:31.860823 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:31.860823 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:31.862919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:31.862919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:31.862919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:31.862919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:31.862919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:31.862919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:31.862919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:31.862919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:31.862919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:31.862919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:31.862919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:31.862919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:08:31.866111 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 131/254] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:31.867440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf606f3d-72db-4600-9bd7-2de1b1c39975 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:31.902094 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a370ba-6894-4a6b-b04e-6907c5c55147 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:31.953845 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452155e-4847-467f-9e18-2b61c4c40c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:31.954864 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452155e-4847-467f-9e18-2b61c4c40c98 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:31.956594 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452155e-4847-467f-9e18-2b61c4c40c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:31.957079 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452155e-4847-467f-9e18-2b61c4c40c98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:31.984531 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf606f3d-72db-4600-9bd7-2de1b1c39975 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:32.021992 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5143d7d5-49cb-45b5-b266-76dada66e31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:32.022650 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143d7d5-49cb-45b5-b266-76dada66e31c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:32.022916 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143d7d5-49cb-45b5-b266-76dada66e31c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:32.023192 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143d7d5-49cb-45b5-b266-76dada66e31c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:32.031941 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:32.059344 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a452155e-4847-467f-9e18-2b61c4c40c98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:32.062262 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a452155e-4847-467f-9e18-2b61c4c40c98 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:32.105839 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5143d7d5-49cb-45b5-b266-76dada66e31c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:32.108186 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5143d7d5-49cb-45b5-b266-76dada66e31c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:32.109003 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:32.196221 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 124/255] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:08:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:08:32.196221 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452155e-4847-467f-9e18-2b61c4c40c98 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:32.267740 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 132/256] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:08:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:08:32.269901 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5143d7d5-49cb-45b5-b266-76dada66e31c None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:32.310261 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452155e-4847-467f-9e18-2b61c4c40c98 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:32.326043 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:32.362859 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0cb5df-92a0-4eee-9622-4fca33ecc34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:32.363499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0cb5df-92a0-4eee-9622-4fca33ecc34b None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:32.363779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0cb5df-92a0-4eee-9622-4fca33ecc34b None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:32.364049 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0cb5df-92a0-4eee-9622-4fca33ecc34b None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:32.364780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0cb5df-92a0-4eee-9622-4fca33ecc34b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:32.365081 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0cb5df-92a0-4eee-9622-4fca33ecc34b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:32.366518 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0cb5df-92a0-4eee-9622-4fca33ecc34b None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:32.378902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 125/257] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:32.380510 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0cb5df-92a0-4eee-9622-4fca33ecc34b None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:32.386160 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:32.405810 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5143d7d5-49cb-45b5-b266-76dada66e31c None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:32.445027 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a4aee-f1af-451f-84a2-1254a245e5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:32.446458 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a4aee-f1af-451f-84a2-1254a245e5b0 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:32.447711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a4aee-f1af-451f-84a2-1254a245e5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:32.448451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a4aee-f1af-451f-84a2-1254a245e5b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:32.449366 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570a4aee-f1af-451f-84a2-1254a245e5b0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:32.450115 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570a4aee-f1af-451f-84a2-1254a245e5b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:32.452421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570a4aee-f1af-451f-84a2-1254a245e5b0 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:32.464323 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 133/258] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 27 04:08:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:32.464875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a4aee-f1af-451f-84a2-1254a245e5b0 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:32.521886 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0cb5df-92a0-4eee-9622-4fca33ecc34b None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:32.558561 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186d6a2-25b8-4485-8634-5a127408ab89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:32.559509 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186d6a2-25b8-4485-8634-5a127408ab89 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:32.559951 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186d6a2-25b8-4485-8634-5a127408ab89 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:32.560416 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186d6a2-25b8-4485-8634-5a127408ab89 None admin] PATH_INFO: `/v3/projects/932a11c28fa2405793ce99d7097c63a5` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:32.561710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186d6a2-25b8-4485-8634-5a127408ab89 None admin] RBAC: Authorizing `identity:get_project(project_id=932a11c28fa2405793ce99d7097c63a5)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:32.563692 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186d6a2-25b8-4485-8634-5a127408ab89 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:32.566388 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 126/259] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:32 2022] GET /identity/v3/projects/932a11c28fa2405793ce99d7097c63a5 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:32.567290 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186d6a2-25b8-4485-8634-5a127408ab89 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:32.591596 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:32.596895 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a4aee-f1af-451f-84a2-1254a245e5b0 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:32.640722 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72944ed5-043f-41ff-a3b9-86f6827f2bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:32.640906 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:32.642421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72944ed5-043f-41ff-a3b9-86f6827f2bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:32.643258 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72944ed5-043f-41ff-a3b9-86f6827f2bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:32.643626 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72944ed5-043f-41ff-a3b9-86f6827f2bc2 None admin] PATH_INFO: `/v3/projects/932a11c28fa2405793ce99d7097c63a5` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:32.645133 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72944ed5-043f-41ff-a3b9-86f6827f2bc2 None admin] RBAC: Authorizing `identity:get_project(project_id=932a11c28fa2405793ce99d7097c63a5)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:32.646663 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72944ed5-043f-41ff-a3b9-86f6827f2bc2 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:32.648351 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 134/260] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:32 2022] GET /identity/v3/projects/932a11c28fa2405793ce99d7097c63a5 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:32.649111 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72944ed5-043f-41ff-a3b9-86f6827f2bc2 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:32.684626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186d6a2-25b8-4485-8634-5a127408ab89 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:32.740018 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3f1821-0d6b-4eb8-9f13-18a0997a2957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:32.741463 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f1821-0d6b-4eb8-9f13-18a0997a2957 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:32.742125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f1821-0d6b-4eb8-9f13-18a0997a2957 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:32.742687 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f1821-0d6b-4eb8-9f13-18a0997a2957 None admin] PATH_INFO: `/v3/users/1130e205d84d42a796a43c0153f35b0e` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:32.744368 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3f1821-0d6b-4eb8-9f13-18a0997a2957 None admin] RBAC: Authorizing `identity:get_user(user_id=1130e205d84d42a796a43c0153f35b0e)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:32.746886 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3f1821-0d6b-4eb8-9f13-18a0997a2957 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:32.749392 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 127/261] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:32 2022] GET /identity/v3/users/1130e205d84d42a796a43c0153f35b0e => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:32.755011 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3f1821-0d6b-4eb8-9f13-18a0997a2957 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:32.768488 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72944ed5-043f-41ff-a3b9-86f6827f2bc2 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:32.803926 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ed2bd-3fa3-4cac-b04a-bcd7b0e60033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:32.804498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ed2bd-3fa3-4cac-b04a-bcd7b0e60033 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:32.805030 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ed2bd-3fa3-4cac-b04a-bcd7b0e60033 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:32.805030 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ed2bd-3fa3-4cac-b04a-bcd7b0e60033 None admin] PATH_INFO: `/v3/users/933e108d6dff4162a70ed6be5436900b` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:32.806253 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ed2bd-3fa3-4cac-b04a-bcd7b0e60033 None admin] RBAC: Authorizing `identity:get_user(user_id=933e108d6dff4162a70ed6be5436900b)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:32.807697 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ed2bd-3fa3-4cac-b04a-bcd7b0e60033 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:32.814060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 135/262] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:32 2022] GET /identity/v3/users/933e108d6dff4162a70ed6be5436900b => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:32.814060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ed2bd-3fa3-4cac-b04a-bcd7b0e60033 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:32.848472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:32.871674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3f1821-0d6b-4eb8-9f13-18a0997a2957 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:32.907216 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9401985e-3400-4d53-9167-b3eae3d2e8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:32.908751 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401985e-3400-4d53-9167-b3eae3d2e8d8 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:32.909239 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401985e-3400-4d53-9167-b3eae3d2e8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:32.909709 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401985e-3400-4d53-9167-b3eae3d2e8d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:32.910761 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401985e-3400-4d53-9167-b3eae3d2e8d8 None admin] RBAC: Adding query filter params (role.id=84fc927928374b91847a13b7fbb8ddd1, scope.project.id=932a11c28fa2405793ce99d7097c63a5, user.id=ef7b73952eec4ef6b1a0688fce3b4dd4) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:32.911270 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401985e-3400-4d53-9167-b3eae3d2e8d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:32.913020 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401985e-3400-4d53-9167-b3eae3d2e8d8 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:32.918187 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ed2bd-3fa3-4cac-b04a-bcd7b0e60033 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:32.919725 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 128/263] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:08:32 2022] GET /identity/v3/role_assignments?user.id=ef7b73952eec4ef6b1a0688fce3b4dd4&scope.project.id=932a11c28fa2405793ce99d7097c63a5&role.id=84fc927928374b91847a13b7fbb8ddd1 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:32.920777 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9401985e-3400-4d53-9167-b3eae3d2e8d8 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:32.924781 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:32.957340 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b664f-5e72-4c4e-a7dc-2d3693cc4f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:32.958038 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b664f-5e72-4c4e-a7dc-2d3693cc4f77 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:32.958324 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b664f-5e72-4c4e-a7dc-2d3693cc4f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:32.958644 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b664f-5e72-4c4e-a7dc-2d3693cc4f77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:32.959518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b664f-5e72-4c4e-a7dc-2d3693cc4f77 None admin] RBAC: Adding query filter params (role.id=cdfc87279ea24b65bae3961035133725, scope.project.id=932a11c28fa2405793ce99d7097c63a5, user.id=1130e205d84d42a796a43c0153f35b0e) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:32.959748 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b664f-5e72-4c4e-a7dc-2d3693cc4f77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:32.961198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b664f-5e72-4c4e-a7dc-2d3693cc4f77 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:32.967966 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 136/264] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:08:32 2022] GET /identity/v3/role_assignments?user.id=1130e205d84d42a796a43c0153f35b0e&scope.project.id=932a11c28fa2405793ce99d7097c63a5&role.id=cdfc87279ea24b65bae3961035133725 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:32.968846 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b664f-5e72-4c4e-a7dc-2d3693cc4f77 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:33.039645 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9401985e-3400-4d53-9167-b3eae3d2e8d8 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:33.081112 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e53b9-c1a5-4faf-8062-7c0a53dd01ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:33.082177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e53b9-c1a5-4faf-8062-7c0a53dd01ee None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:33.082658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e53b9-c1a5-4faf-8062-7c0a53dd01ee None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:33.083125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e53b9-c1a5-4faf-8062-7c0a53dd01ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:33.123020 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:33.127392 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b664f-5e72-4c4e-a7dc-2d3693cc4f77 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:33.162253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273e53b9-c1a5-4faf-8062-7c0a53dd01ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:33.164150 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273e53b9-c1a5-4faf-8062-7c0a53dd01ee None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:33.167938 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:33.168106 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd44eb1-57a2-4c91-aab7-edd94fa81740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:33.169004 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd44eb1-57a2-4c91-aab7-edd94fa81740 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:33.169004 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd44eb1-57a2-4c91-aab7-edd94fa81740 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:33.169288 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd44eb1-57a2-4c91-aab7-edd94fa81740 None admin] PATH_INFO: `/v3/projects/932a11c28fa2405793ce99d7097c63a5` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:33.171057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd44eb1-57a2-4c91-aab7-edd94fa81740 None admin] RBAC: Authorizing `identity:get_project(project_id=932a11c28fa2405793ce99d7097c63a5)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:33.172672 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd44eb1-57a2-4c91-aab7-edd94fa81740 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:33.175410 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 137/265] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:32 2022] GET /identity/v3/projects/932a11c28fa2405793ce99d7097c63a5 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:33.185624 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd44eb1-57a2-4c91-aab7-edd94fa81740 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:33.334076 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 129/266] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:08:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:08:33.341189 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e53b9-c1a5-4faf-8062-7c0a53dd01ee None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:33.378527 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd44eb1-57a2-4c91-aab7-edd94fa81740 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:33.439446 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b1599-0120-425d-ab75-d6fd40b514c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:33.440641 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b1599-0120-425d-ab75-d6fd40b514c7 None admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:33.440641 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b1599-0120-425d-ab75-d6fd40b514c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:33.441047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b1599-0120-425d-ab75-d6fd40b514c7 None admin] PATH_INFO: `/v3/projects/932a11c28fa2405793ce99d7097c63a5/users/933e108d6dff4162a70ed6be5436900b/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:33.443161 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77b1599-0120-425d-ab75-d6fd40b514c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=932a11c28fa2405793ce99d7097c63a5, user_id=933e108d6dff4162a70ed6be5436900b, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:33.444729 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77b1599-0120-425d-ab75-d6fd40b514c7 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:33.489325 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 138/267] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 27 04:08:33 2022] PUT /identity/v3/projects/932a11c28fa2405793ce99d7097c63a5/users/933e108d6dff4162a70ed6be5436900b/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:08:33.500308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef304204-849b-43e3-8176-57b34182e5b4 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:33.500643 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef304204-849b-43e3-8176-57b34182e5b4 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:33.500858 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef304204-849b-43e3-8176-57b34182e5b4 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:33.505732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 139/268] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:33.511626 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf314a-da85-4393-adcb-7901c1c991a8 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:33.511812 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf314a-da85-4393-adcb-7901c1c991a8 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:33.512077 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf314a-da85-4393-adcb-7901c1c991a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:33.550800 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-3bcf314a-da85-4393-adcb-7901c1c991a8 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:33.551481 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e53b9-c1a5-4faf-8062-7c0a53dd01ee None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:33.552842 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-3bcf314a-da85-4393-adcb-7901c1c991a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:33.557529 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:33.596149 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:33.621455 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72fc4a-f450-48e8-bcfd-7d4867ac1215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:33.622164 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72fc4a-f450-48e8-bcfd-7d4867ac1215 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:33.622443 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72fc4a-f450-48e8-bcfd-7d4867ac1215 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:33.622714 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72fc4a-f450-48e8-bcfd-7d4867ac1215 None admin] PATH_INFO: `/v3/projects/932a11c28fa2405793ce99d7097c63a5` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:33.623913 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f72fc4a-f450-48e8-bcfd-7d4867ac1215 None admin] RBAC: Authorizing `identity:get_project(project_id=932a11c28fa2405793ce99d7097c63a5)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:33.625508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f72fc4a-f450-48e8-bcfd-7d4867ac1215 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:33.628176 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 130/269] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:33 2022] GET /identity/v3/projects/932a11c28fa2405793ce99d7097c63a5 => generated 302 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:08:33.629346 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e50d8-e1ae-43bf-8f1a-785f6e5ba338 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:33.629624 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e50d8-e1ae-43bf-8f1a-785f6e5ba338 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:33.629894 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e50d8-e1ae-43bf-8f1a-785f6e5ba338 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:33.636554 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 131/270] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:33.637518 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e50d8-e1ae-43bf-8f1a-785f6e5ba338 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:33.684941 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 140/271] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:33.686549 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf66151-3401-44d0-bfa2-028ae5d3fbb5 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:33.686959 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf66151-3401-44d0-bfa2-028ae5d3fbb5 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:33.687245 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf66151-3401-44d0-bfa2-028ae5d3fbb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:33.709371 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-ccf66151-3401-44d0-bfa2-028ae5d3fbb5 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:33.711334 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-ccf66151-3401-44d0-bfa2-028ae5d3fbb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:33.775595 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e50d8-e1ae-43bf-8f1a-785f6e5ba338 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:33.805336 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 141/272] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:33.810340 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf66151-3401-44d0-bfa2-028ae5d3fbb5 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:33.815320 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc2406f-7806-4b84-a2c4-03812e6e6d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:33.816033 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc2406f-7806-4b84-a2c4-03812e6e6d91 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:33.816309 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc2406f-7806-4b84-a2c4-03812e6e6d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:33.816576 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc2406f-7806-4b84-a2c4-03812e6e6d91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:33.817381 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc2406f-7806-4b84-a2c4-03812e6e6d91 None admin] RBAC: Adding query filter params (role.id=cf771a8e94ee46149eb081347fea6d67, scope.project.id=932a11c28fa2405793ce99d7097c63a5, user.id=1130e205d84d42a796a43c0153f35b0e) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:33.817684 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc2406f-7806-4b84-a2c4-03812e6e6d91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:33.819180 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc2406f-7806-4b84-a2c4-03812e6e6d91 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:33.828528 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 132/273] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:08:33 2022] GET /identity/v3/role_assignments?user.id=1130e205d84d42a796a43c0153f35b0e&scope.project.id=932a11c28fa2405793ce99d7097c63a5&role.id=cf771a8e94ee46149eb081347fea6d67 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:33.883251 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc2406f-7806-4b84-a2c4-03812e6e6d91 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:33.954353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:33.960801 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:33.966085 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf66151-3401-44d0-bfa2-028ae5d3fbb5 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:34.048763 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad70d4c9-f3a0-4ff8-8445-3ca331d180c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:34.049487 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad70d4c9-f3a0-4ff8-8445-3ca331d180c3 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:34.049747 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad70d4c9-f3a0-4ff8-8445-3ca331d180c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:34.050043 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad70d4c9-f3a0-4ff8-8445-3ca331d180c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:34.052057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad70d4c9-f3a0-4ff8-8445-3ca331d180c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:34.053761 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad70d4c9-f3a0-4ff8-8445-3ca331d180c3 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:34.057291 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 142/274] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:08:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:34.070266 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad70d4c9-f3a0-4ff8-8445-3ca331d180c3 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:34.114276 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc2406f-7806-4b84-a2c4-03812e6e6d91 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:34.157281 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3501b-95d4-4d69-87db-a8398ce555e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:34.159282 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3501b-95d4-4d69-87db-a8398ce555e3 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:34.159886 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3501b-95d4-4d69-87db-a8398ce555e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:34.160255 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3501b-95d4-4d69-87db-a8398ce555e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:34.179388 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3501b-95d4-4d69-87db-a8398ce555e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:34.185159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-e4e3501b-95d4-4d69-87db-a8398ce555e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:34.185159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:34.185159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:34.185159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:34.185159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:34.185159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:34.185159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:34.185159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:34.185159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:34.185159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:34.185159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:34.185159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:34.185159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:34.185159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:34.185159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:34.185159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:34.186050 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:34.186050 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:34.186050 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:34.186050 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:34.186050 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:34.186050 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:34.186050 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:34.186050 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:34.186050 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:34.186050 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:34.186050 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:34.186050 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:34.186050 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:34.186050 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:34.186050 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:34.186050 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:34.186829 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:34.186829 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:34.186829 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:34.186829 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:34.186829 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:34.186829 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:34.186829 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:34.186829 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:34.186829 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:34.186829 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:34.186829 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:34.186829 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:08:34.186829 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 133/275] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:34.194439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3501b-95d4-4d69-87db-a8398ce555e3 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:34.201119 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad70d4c9-f3a0-4ff8-8445-3ca331d180c3 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:34.248392 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d320dca4-6179-49be-9949-43869155fede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:34.250211 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320dca4-6179-49be-9949-43869155fede None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:34.250712 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320dca4-6179-49be-9949-43869155fede None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:34.251188 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320dca4-6179-49be-9949-43869155fede None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:34.252678 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d320dca4-6179-49be-9949-43869155fede None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:34.258033 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d320dca4-6179-49be-9949-43869155fede None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:34.285923 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 143/276] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 27 04:08:34 2022] POST /identity/v3/users => generated 293 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:08:34.346372 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:34.353208 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3501b-95d4-4d69-87db-a8398ce555e3 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:34.392447 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc88113-df17-4b75-afe6-ff33078eb8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:34.393782 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc88113-df17-4b75-afe6-ff33078eb8b6 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:34.394112 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc88113-df17-4b75-afe6-ff33078eb8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:34.394389 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc88113-df17-4b75-afe6-ff33078eb8b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:34.395133 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc88113-df17-4b75-afe6-ff33078eb8b6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:34.395437 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc88113-df17-4b75-afe6-ff33078eb8b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:34.396960 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc88113-df17-4b75-afe6-ff33078eb8b6 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:34.407357 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 134/277] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 27 04:08:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:34.413732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d320dca4-6179-49be-9949-43869155fede None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:34.431422 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:34.543211 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d320dca4-6179-49be-9949-43869155fede None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:34.577827 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122dd7cb-8911-436e-8db6-336047ed0da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:34.578529 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dd7cb-8911-436e-8db6-336047ed0da7 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:34.578809 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dd7cb-8911-436e-8db6-336047ed0da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:34.579084 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dd7cb-8911-436e-8db6-336047ed0da7 None admin] PATH_INFO: `/v3/users/60577426a96040768a73e7119552bb6d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:34.592988 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122dd7cb-8911-436e-8db6-336047ed0da7 None admin] RBAC: Authorizing `identity:get_user(user_id=60577426a96040768a73e7119552bb6d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:34.594480 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122dd7cb-8911-436e-8db6-336047ed0da7 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:34.596999 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 144/278] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:34 2022] GET /identity/v3/users/60577426a96040768a73e7119552bb6d => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:34.602939 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc88113-df17-4b75-afe6-ff33078eb8b6 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:34.723626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc88113-df17-4b75-afe6-ff33078eb8b6 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:34.759340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3caf2e-7e36-4884-8aca-1691b87177e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:34.760026 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3caf2e-7e36-4884-8aca-1691b87177e1 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:34.760306 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3caf2e-7e36-4884-8aca-1691b87177e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:34.760581 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3caf2e-7e36-4884-8aca-1691b87177e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:34.761425 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3caf2e-7e36-4884-8aca-1691b87177e1 None admin] RBAC: Adding query filter params (role.id=7b6f2ade82434ee79175385de17049eb, user.id=60577426a96040768a73e7119552bb6d, scope.system=all) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:34.761737 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3caf2e-7e36-4884-8aca-1691b87177e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:34.763227 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3caf2e-7e36-4884-8aca-1691b87177e1 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:34.769486 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 135/279] 10.222.0.46 () {56 vars in 1308 bytes} [Wed Jul 27 04:08:34 2022] GET /identity/v3/role_assignments?user.id=60577426a96040768a73e7119552bb6d&scope.system=all&role.id=7b6f2ade82434ee79175385de17049eb => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:35.519515 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f1d77-c38a-457c-8cba-be0bbb96aefa None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:35.519836 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f1d77-c38a-457c-8cba-be0bbb96aefa None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:35.520069 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f1d77-c38a-457c-8cba-be0bbb96aefa None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:35.521715 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 145/280] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:35.526718 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb58802-0e4d-4baa-842b-d010e8931929 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:35.527006 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb58802-0e4d-4baa-842b-d010e8931929 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:35.527284 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb58802-0e4d-4baa-842b-d010e8931929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:35.555346 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-6bb58802-0e4d-4baa-842b-d010e8931929 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:35.556507 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-6bb58802-0e4d-4baa-842b-d010e8931929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:35.581460 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:35.651870 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 136/281] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:35.691547 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f1d77-c38a-457c-8cba-be0bbb96aefa None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:35.780199 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:35.831235 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f1d77-c38a-457c-8cba-be0bbb96aefa None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:35.880388 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c341e5-22f1-4f56-be41-47912757ad11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:35.881078 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c341e5-22f1-4f56-be41-47912757ad11 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:35.881353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c341e5-22f1-4f56-be41-47912757ad11 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:35.881626 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c341e5-22f1-4f56-be41-47912757ad11 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:35.897156 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c341e5-22f1-4f56-be41-47912757ad11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:35.902132 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-20c341e5-22f1-4f56-be41-47912757ad11 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 04:08:35.902132 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:35.902132 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:35.902132 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:35.902132 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:35.902132 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:35.902132 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:35.902132 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:35.902132 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:35.902132 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:35.902132 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:35.902132 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:35.902132 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:35.902132 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:35.902132 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:35.902132 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:35.902969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:35.902969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:35.902969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:35.902969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:35.902969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:35.902969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:35.902969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:35.902969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:35.902969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:35.902969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:35.902969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:35.902969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:35.902969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:35.902969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:35.902969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:35.902969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:35.903744 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:35.903744 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:35.903744 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:35.903744 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:35.903744 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:35.903744 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:35.903744 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:35.903744 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:35.903744 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:35.903744 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:35.903744 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 04:08:35.903744 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:08:35.903744 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 146/282] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:35.910718 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb58802-0e4d-4baa-842b-d010e8931929 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:35.975168 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edc8a4-beb1-4931-a024-37e6f391d040 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:35.975508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edc8a4-beb1-4931-a024-37e6f391d040 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:35.975728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edc8a4-beb1-4931-a024-37e6f391d040 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:35.977335 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 147/283] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:35.995112 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c77901-0a72-4e65-aeff-0302b4f2e1d9 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:35.995395 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c77901-0a72-4e65-aeff-0302b4f2e1d9 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:35.995672 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c77901-0a72-4e65-aeff-0302b4f2e1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:36.043044 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-71c77901-0a72-4e65-aeff-0302b4f2e1d9 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:36.044185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-71c77901-0a72-4e65-aeff-0302b4f2e1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:36.065482 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb58802-0e4d-4baa-842b-d010e8931929 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:36.102535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63194cf5-3db6-4890-98dd-374a1c5bb606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:36.103518 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63194cf5-3db6-4890-98dd-374a1c5bb606 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:36.103994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63194cf5-3db6-4890-98dd-374a1c5bb606 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:36.104462 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63194cf5-3db6-4890-98dd-374a1c5bb606 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:36.105370 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63194cf5-3db6-4890-98dd-374a1c5bb606 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:36.105882 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63194cf5-3db6-4890-98dd-374a1c5bb606 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:36.107607 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63194cf5-3db6-4890-98dd-374a1c5bb606 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:36.116770 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 137/284] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 27 04:08:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:36.122995 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63194cf5-3db6-4890-98dd-374a1c5bb606 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:36.140017 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:36.223295 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 148/285] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 04:08:36.232640 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63194cf5-3db6-4890-98dd-374a1c5bb606 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:36.281536 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce3cb0-04eb-45b6-a11b-5aad505229d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:36.282562 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce3cb0-04eb-45b6-a11b-5aad505229d1 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:36.283067 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce3cb0-04eb-45b6-a11b-5aad505229d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:36.283619 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce3cb0-04eb-45b6-a11b-5aad505229d1 None admin] PATH_INFO: `/v3/users/ef7b73952eec4ef6b1a0688fce3b4dd4` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:36.284990 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce3cb0-04eb-45b6-a11b-5aad505229d1 None admin] RBAC: Authorizing `identity:get_user(user_id=ef7b73952eec4ef6b1a0688fce3b4dd4)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:36.286689 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce3cb0-04eb-45b6-a11b-5aad505229d1 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:36.288794 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 138/286] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:36 2022] GET /identity/v3/users/ef7b73952eec4ef6b1a0688fce3b4dd4 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:36.294174 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce3cb0-04eb-45b6-a11b-5aad505229d1 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:36.294319 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c77901-0a72-4e65-aeff-0302b4f2e1d9 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:36.401173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce3cb0-04eb-45b6-a11b-5aad505229d1 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:36.422383 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:36.438872 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b623aa-514d-4555-9962-6cc485d63e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:36.440112 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b623aa-514d-4555-9962-6cc485d63e85 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:36.440634 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b623aa-514d-4555-9962-6cc485d63e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:36.441103 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b623aa-514d-4555-9962-6cc485d63e85 None admin] PATH_INFO: `/v3/projects/932a11c28fa2405793ce99d7097c63a5` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:36.442639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b623aa-514d-4555-9962-6cc485d63e85 None admin] RBAC: Authorizing `identity:get_project(project_id=932a11c28fa2405793ce99d7097c63a5)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:36.444585 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b623aa-514d-4555-9962-6cc485d63e85 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:36.446530 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 139/287] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:36 2022] GET /identity/v3/projects/932a11c28fa2405793ce99d7097c63a5 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:36.454141 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b623aa-514d-4555-9962-6cc485d63e85 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:36.474000 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:36.557557 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c77901-0a72-4e65-aeff-0302b4f2e1d9 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:36.621718 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b623aa-514d-4555-9962-6cc485d63e85 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:36.646689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec06ff5-0bb5-4837-b7df-b9e49f144642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:36.647361 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec06ff5-0bb5-4837-b7df-b9e49f144642 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:36.647648 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec06ff5-0bb5-4837-b7df-b9e49f144642 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:36.647928 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec06ff5-0bb5-4837-b7df-b9e49f144642 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:36.660246 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c6c8f-76a5-417e-aed0-c93da0cfa523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:36.661559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c6c8f-76a5-417e-aed0-c93da0cfa523 None admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:36.661991 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c6c8f-76a5-417e-aed0-c93da0cfa523 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:36.662419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c6c8f-76a5-417e-aed0-c93da0cfa523 None admin] PATH_INFO: `/v3/projects/932a11c28fa2405793ce99d7097c63a5/users/ef7b73952eec4ef6b1a0688fce3b4dd4/roles/84fc927928374b91847a13b7fbb8ddd1` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:36.664479 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c6c8f-76a5-417e-aed0-c93da0cfa523 None admin] RBAC: Authorizing `identity:create_grant(project_id=932a11c28fa2405793ce99d7097c63a5, user_id=ef7b73952eec4ef6b1a0688fce3b4dd4, role_id=84fc927928374b91847a13b7fbb8ddd1)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:36.666081 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c6c8f-76a5-417e-aed0-c93da0cfa523 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:36.675447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec06ff5-0bb5-4837-b7df-b9e49f144642 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:36.687818 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 140/288] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 27 04:08:36 2022] PUT /identity/v3/projects/932a11c28fa2405793ce99d7097c63a5/users/ef7b73952eec4ef6b1a0688fce3b4dd4/roles/84fc927928374b91847a13b7fbb8ddd1 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:08:36.688482 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-4ec06ff5-0bb5-4837-b7df-b9e49f144642 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:36.688482 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:36.688482 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:36.688482 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:36.688482 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:36.688482 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:36.688482 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:36.688482 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:36.688482 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:36.688482 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:36.688482 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:36.688482 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:36.688482 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:36.688482 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:36.688482 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:36.688482 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:36.689328 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:36.689328 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:36.689328 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:36.689328 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:36.689328 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:36.689328 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:36.689328 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:36.689328 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:36.689328 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:36.689328 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:36.689328 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:36.689328 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:36.689328 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:36.689328 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:36.689328 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:36.689328 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:36.690703 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:36.690703 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:36.690703 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:36.690703 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:36.690703 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:36.690703 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:36.690703 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:36.690703 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:36.690703 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:36.690703 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:36.690703 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:36.690703 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:08:36.693635 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 149/289] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:36.700530 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c6c8f-76a5-417e-aed0-c93da0cfa523 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:36.716472 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:36.815207 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c6c8f-76a5-417e-aed0-c93da0cfa523 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:36.851746 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a532eb-5d79-478b-b09f-75dbf8784e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:36.852705 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a532eb-5d79-478b-b09f-75dbf8784e18 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:36.853165 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a532eb-5d79-478b-b09f-75dbf8784e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:36.853612 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a532eb-5d79-478b-b09f-75dbf8784e18 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:36.854653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a532eb-5d79-478b-b09f-75dbf8784e18 None admin] RBAC: Adding query filter params (name=member) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:36.855135 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a532eb-5d79-478b-b09f-75dbf8784e18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:36.856746 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a532eb-5d79-478b-b09f-75dbf8784e18 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:36.866880 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 141/290] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 27 04:08:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:36.872713 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec06ff5-0bb5-4837-b7df-b9e49f144642 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:36.911413 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d006aaaa-f018-4d68-b0c5-a3953e1e974b None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:36.912077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d006aaaa-f018-4d68-b0c5-a3953e1e974b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:36.912528 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d006aaaa-f018-4d68-b0c5-a3953e1e974b None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:36.914397 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 142/291] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 04:08:36.919781 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d515b-d3a0-4a2b-be82-d2e534ee7a47 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:36.920323 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d515b-d3a0-4a2b-be82-d2e534ee7a47 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:36.920798 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d515b-d3a0-4a2b-be82-d2e534ee7a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:36.956577 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-138d515b-d3a0-4a2b-be82-d2e534ee7a47 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:36.957808 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-138d515b-d3a0-4a2b-be82-d2e534ee7a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:37.014382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec06ff5-0bb5-4837-b7df-b9e49f144642 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:37.035194 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 143/292] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:37.036577 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0e1ee-d3e0-48ee-b13b-cb86ea183fef None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:37.036939 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0e1ee-d3e0-48ee-b13b-cb86ea183fef None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:37.037301 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0e1ee-d3e0-48ee-b13b-cb86ea183fef None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:37.042053 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:37.045602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 144/293] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:37 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:37.059297 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22758d8-ce57-4701-8f5a-e3c1e38b4733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:37.059504 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435284bb-f853-4a35-a481-808f684364b7 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:37.059733 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435284bb-f853-4a35-a481-808f684364b7 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:37.059931 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435284bb-f853-4a35-a481-808f684364b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:37.060026 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22758d8-ce57-4701-8f5a-e3c1e38b4733 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:37.060207 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22758d8-ce57-4701-8f5a-e3c1e38b4733 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:37.060480 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22758d8-ce57-4701-8f5a-e3c1e38b4733 None admin] PATH_INFO: `/v3/users/933e108d6dff4162a70ed6be5436900b` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:37.061647 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22758d8-ce57-4701-8f5a-e3c1e38b4733 None admin] RBAC: Authorizing `identity:get_user(user_id=933e108d6dff4162a70ed6be5436900b)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:37.063395 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22758d8-ce57-4701-8f5a-e3c1e38b4733 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:37.065859 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 150/294] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:36 2022] GET /identity/v3/users/933e108d6dff4162a70ed6be5436900b => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:37.070395 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:37.071499 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22758d8-ce57-4701-8f5a-e3c1e38b4733 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:37.085188 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-435284bb-f853-4a35-a481-808f684364b7 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:37.086409 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-435284bb-f853-4a35-a481-808f684364b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:37.162856 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 145/295] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 04:08:37.163169 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435284bb-f853-4a35-a481-808f684364b7 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:37.200782 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22758d8-ce57-4701-8f5a-e3c1e38b4733 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:37.243204 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08afd4c9-cd46-49a9-af06-2fc503c6b832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:37.244581 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afd4c9-cd46-49a9-af06-2fc503c6b832 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:37.245042 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afd4c9-cd46-49a9-af06-2fc503c6b832 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:37.245491 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afd4c9-cd46-49a9-af06-2fc503c6b832 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:37.262231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08afd4c9-cd46-49a9-af06-2fc503c6b832 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:37.270784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-08afd4c9-cd46-49a9-af06-2fc503c6b832 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 04:08:37.270784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:37.270784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:37.270784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:37.270784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:37.270784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:37.270784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:37.270784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:37.270784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:37.270784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:37.270784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:37.270784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:37.270784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:37.270784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:37.270784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:37.270784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:37.271673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:37.271673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:37.271673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:37.271673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:37.271673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:37.271673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:37.271673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:37.271673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:37.271673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:37.271673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:37.271673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:37.271673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:37.271673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:37.271673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:37.271673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:37.271673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:37.272459 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:37.272459 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:37.272459 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:37.272459 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:37.272459 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:37.272459 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:37.272459 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:37.272459 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:37.272459 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:37.272459 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:37.272459 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 04:08:37.272459 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:08:37.274860 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 151/296] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:37.275787 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08afd4c9-cd46-49a9-af06-2fc503c6b832 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:37.276783 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435284bb-f853-4a35-a481-808f684364b7 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:37.316724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:37.323355 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532ee18-1818-4f08-bed4-dc1934f04316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:37.324629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532ee18-1818-4f08-bed4-dc1934f04316 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:37.325054 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532ee18-1818-4f08-bed4-dc1934f04316 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:37.325491 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532ee18-1818-4f08-bed4-dc1934f04316 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:37.334659 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:37.392673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08afd4c9-cd46-49a9-af06-2fc503c6b832 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:37.410278 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9532ee18-1818-4f08-bed4-dc1934f04316 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:37.411831 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9532ee18-1818-4f08-bed4-dc1934f04316 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:37.430105 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97247787-02fe-4ede-a6bd-ec3e358853df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:37.430754 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97247787-02fe-4ede-a6bd-ec3e358853df None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:37.431065 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97247787-02fe-4ede-a6bd-ec3e358853df None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:37.431309 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97247787-02fe-4ede-a6bd-ec3e358853df None admin] PATH_INFO: `/v3/domains/default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:37.432447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97247787-02fe-4ede-a6bd-ec3e358853df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:37.433866 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97247787-02fe-4ede-a6bd-ec3e358853df None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:37.435476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 152/297] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:08:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:37.436228 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97247787-02fe-4ede-a6bd-ec3e358853df None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:37.534564 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 146/298] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:08:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:08:37.534564 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad5727-526e-435b-a129-186386d2dfe5 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:37.534564 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad5727-526e-435b-a129-186386d2dfe5 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:37.534564 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad5727-526e-435b-a129-186386d2dfe5 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:37.534564 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 147/299] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:37.534564 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad5727-526e-435b-a129-186386d2dfe5 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:37.550272 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97247787-02fe-4ede-a6bd-ec3e358853df None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:37.568007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:37.573254 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:37.589407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655bedc4-cfc7-49f3-9479-c705c9a27246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:37.590099 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655bedc4-cfc7-49f3-9479-c705c9a27246 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:37.590395 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655bedc4-cfc7-49f3-9479-c705c9a27246 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:37.590672 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655bedc4-cfc7-49f3-9479-c705c9a27246 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:37.591400 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655bedc4-cfc7-49f3-9479-c705c9a27246 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:37.591706 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655bedc4-cfc7-49f3-9479-c705c9a27246 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:37.593080 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655bedc4-cfc7-49f3-9479-c705c9a27246 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:37.602184 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 153/300] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 27 04:08:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:37.602922 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655bedc4-cfc7-49f3-9479-c705c9a27246 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:37.660032 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad5727-526e-435b-a129-186386d2dfe5 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:37.696353 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be28c7e4-6e3d-457b-a87f-afcc7f6fb80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:37.697141 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be28c7e4-6e3d-457b-a87f-afcc7f6fb80c None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:37.697505 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be28c7e4-6e3d-457b-a87f-afcc7f6fb80c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:37.698275 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be28c7e4-6e3d-457b-a87f-afcc7f6fb80c None admin] PATH_INFO: `/v3/groups` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:37.698963 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be28c7e4-6e3d-457b-a87f-afcc7f6fb80c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:37.700412 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be28c7e4-6e3d-457b-a87f-afcc7f6fb80c None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:37.710576 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655bedc4-cfc7-49f3-9479-c705c9a27246 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:37.717713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 148/301] 10.222.0.46 () {60 vars in 1134 bytes} [Wed Jul 27 04:08:37 2022] POST /identity/v3/groups => generated 229 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:08:37.719444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb152a-32da-495c-8b59-75b3b74fa47a None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:37.719952 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb152a-32da-495c-8b59-75b3b74fa47a None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:37.720053 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb152a-32da-495c-8b59-75b3b74fa47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:37.749818 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-76cb152a-32da-495c-8b59-75b3b74fa47a None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:37.751500 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-76cb152a-32da-495c-8b59-75b3b74fa47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:37.758000 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437069e-68f4-450f-a45e-f110bdf1161a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:37.759000 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437069e-68f4-450f-a45e-f110bdf1161a None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:37.759428 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437069e-68f4-450f-a45e-f110bdf1161a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:37.759972 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437069e-68f4-450f-a45e-f110bdf1161a None admin] PATH_INFO: `/v3/projects/932a11c28fa2405793ce99d7097c63a5` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:37.761539 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2437069e-68f4-450f-a45e-f110bdf1161a None admin] RBAC: Authorizing `identity:get_project(project_id=932a11c28fa2405793ce99d7097c63a5)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:37.763097 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2437069e-68f4-450f-a45e-f110bdf1161a None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:37.765269 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 154/302] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:37 2022] GET /identity/v3/projects/932a11c28fa2405793ce99d7097c63a5 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:37.766202 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437069e-68f4-450f-a45e-f110bdf1161a None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:37.846012 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 149/303] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:37.846012 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb152a-32da-495c-8b59-75b3b74fa47a None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:37.847932 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:37.887579 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:37.939260 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437069e-68f4-450f-a45e-f110bdf1161a None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:37.981328 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d940985-e8c1-4b1e-bb5b-d30c76238b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:37.981789 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb152a-32da-495c-8b59-75b3b74fa47a None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:37.982576 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d940985-e8c1-4b1e-bb5b-d30c76238b04 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:37.983068 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d940985-e8c1-4b1e-bb5b-d30c76238b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:37.983688 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d940985-e8c1-4b1e-bb5b-d30c76238b04 None admin] PATH_INFO: `/v3/users/7af85767ac7a490e8c36341b5cc9d5ed` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:37.998049 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d940985-e8c1-4b1e-bb5b-d30c76238b04 None admin] RBAC: Authorizing `identity:get_user(user_id=7af85767ac7a490e8c36341b5cc9d5ed)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:38.000028 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d940985-e8c1-4b1e-bb5b-d30c76238b04 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:38.002386 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 155/304] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:37 2022] GET /identity/v3/users/7af85767ac7a490e8c36341b5cc9d5ed => generated 293 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:38.003539 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d940985-e8c1-4b1e-bb5b-d30c76238b04 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:38.024715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cbd62-6c8d-4c23-bfc0-430f8c893779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:38.025373 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cbd62-6c8d-4c23-bfc0-430f8c893779 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:38.025652 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cbd62-6c8d-4c23-bfc0-430f8c893779 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:38.026000 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cbd62-6c8d-4c23-bfc0-430f8c893779 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:38.026770 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7cbd62-6c8d-4c23-bfc0-430f8c893779 None admin] RBAC: Adding query filter params (role.id=7b6f2ade82434ee79175385de17049eb, scope.project.id=932a11c28fa2405793ce99d7097c63a5, user.id=933e108d6dff4162a70ed6be5436900b) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:38.027082 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7cbd62-6c8d-4c23-bfc0-430f8c893779 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:38.028554 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7cbd62-6c8d-4c23-bfc0-430f8c893779 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:38.036738 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 150/305] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:08:37 2022] GET /identity/v3/role_assignments?user.id=933e108d6dff4162a70ed6be5436900b&scope.project.id=932a11c28fa2405793ce99d7097c63a5&role.id=7b6f2ade82434ee79175385de17049eb => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:38.036738 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cbd62-6c8d-4c23-bfc0-430f8c893779 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:38.209243 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d940985-e8c1-4b1e-bb5b-d30c76238b04 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:38.226338 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:38.234359 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:38.255462 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cbd62-6c8d-4c23-bfc0-430f8c893779 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:38.262888 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8606f887-53a7-4e92-bb5b-f8684ba504c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:38.263541 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606f887-53a7-4e92-bb5b-f8684ba504c6 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:38.263860 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606f887-53a7-4e92-bb5b-f8684ba504c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:38.264191 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606f887-53a7-4e92-bb5b-f8684ba504c6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:38.280592 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8606f887-53a7-4e92-bb5b-f8684ba504c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:38.287416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-8606f887-53a7-4e92-bb5b-f8684ba504c6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:38.287416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:38.287416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:38.287416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:38.287416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:38.287416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:38.287416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:38.287416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:38.287416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:38.287416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:38.287416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:38.287416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:38.287416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:38.287416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:38.287416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:38.287416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:38.288337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:38.288337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:38.288337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:38.288337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:38.288337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:38.288337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:38.288337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:38.288337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:38.288337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:38.288337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:38.288337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:38.288337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:38.288337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:38.288337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:38.288337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:38.288337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:38.289102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:38.289102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:38.289102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:38.289102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:38.289102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:38.289102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:38.289102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:38.289102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:38.289102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:38.289102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:38.289102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:38.289102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:08:38.289102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 156/306] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:38.296061 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8606f887-53a7-4e92-bb5b-f8684ba504c6 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:38.306649 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094b699-c0b3-4fda-82b5-7d1632c4e9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:38.307373 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094b699-c0b3-4fda-82b5-7d1632c4e9d5 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:38.307754 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094b699-c0b3-4fda-82b5-7d1632c4e9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:38.308148 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094b699-c0b3-4fda-82b5-7d1632c4e9d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:38.309024 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d094b699-c0b3-4fda-82b5-7d1632c4e9d5 None admin] RBAC: Adding query filter params (role.id=84fc927928374b91847a13b7fbb8ddd1, user.id=7af85767ac7a490e8c36341b5cc9d5ed, scope.system=all) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:38.309892 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d094b699-c0b3-4fda-82b5-7d1632c4e9d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:38.311934 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d094b699-c0b3-4fda-82b5-7d1632c4e9d5 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:38.320249 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 151/307] 10.222.0.46 () {56 vars in 1308 bytes} [Wed Jul 27 04:08:38 2022] GET /identity/v3/role_assignments?user.id=7af85767ac7a490e8c36341b5cc9d5ed&scope.system=all&role.id=84fc927928374b91847a13b7fbb8ddd1 => generated 226 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:38.438515 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8606f887-53a7-4e92-bb5b-f8684ba504c6 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:38.479015 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e604717f-473a-4b9c-9e4f-15ed474cfe84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:38.479581 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604717f-473a-4b9c-9e4f-15ed474cfe84 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:38.479873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604717f-473a-4b9c-9e4f-15ed474cfe84 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:38.480157 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604717f-473a-4b9c-9e4f-15ed474cfe84 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:38.480910 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e604717f-473a-4b9c-9e4f-15ed474cfe84 None admin] RBAC: Adding query filter params (name=member) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:38.481220 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e604717f-473a-4b9c-9e4f-15ed474cfe84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:38.482669 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e604717f-473a-4b9c-9e4f-15ed474cfe84 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:38.497085 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 157/308] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 27 04:08:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:38.498482 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094b699-c0b3-4fda-82b5-7d1632c4e9d5 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:38.615512 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094b699-c0b3-4fda-82b5-7d1632c4e9d5 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:38.651947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b0d59-1e61-48c7-bda2-dc3ef88f9f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:38.652969 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b0d59-1e61-48c7-bda2-dc3ef88f9f92 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:38.653453 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b0d59-1e61-48c7-bda2-dc3ef88f9f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:38.653949 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b0d59-1e61-48c7-bda2-dc3ef88f9f92 None admin] PATH_INFO: `/v3/users/60577426a96040768a73e7119552bb6d` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:38.655411 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4b0d59-1e61-48c7-bda2-dc3ef88f9f92 None admin] RBAC: Authorizing `identity:get_user(user_id=60577426a96040768a73e7119552bb6d)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:38.657059 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4b0d59-1e61-48c7-bda2-dc3ef88f9f92 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:38.659387 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 152/309] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:38 2022] GET /identity/v3/users/60577426a96040768a73e7119552bb6d => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:38.665997 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e604717f-473a-4b9c-9e4f-15ed474cfe84 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:38.695859 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:38.772961 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e604717f-473a-4b9c-9e4f-15ed474cfe84 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:38.807660 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff2b241-b894-4257-825d-2c79679172fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:38.808340 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2b241-b894-4257-825d-2c79679172fd None admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:38.808627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2b241-b894-4257-825d-2c79679172fd None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:38.808906 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2b241-b894-4257-825d-2c79679172fd None admin] PATH_INFO: `/v3/system/users/60577426a96040768a73e7119552bb6d/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:38.810430 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff2b241-b894-4257-825d-2c79679172fd None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=60577426a96040768a73e7119552bb6d, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:38.811959 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff2b241-b894-4257-825d-2c79679172fd None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:38.830837 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 158/310] 10.222.0.46 () {58 vars in 1334 bytes} [Wed Jul 27 04:08:38 2022] PUT /identity/v3/system/users/60577426a96040768a73e7119552bb6d/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:08:38.991004 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52454a-6947-4c0c-9527-4574d3bd5c39 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:38.991994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52454a-6947-4c0c-9527-4574d3bd5c39 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:38.992488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52454a-6947-4c0c-9527-4574d3bd5c39 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:38.994095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 153/311] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:38.999189 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302b831-9940-429a-abf7-13c51a6c0864 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:38.999490 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302b831-9940-429a-abf7-13c51a6c0864 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:38.999774 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302b831-9940-429a-abf7-13c51a6c0864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:39.019636 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-c302b831-9940-429a-abf7-13c51a6c0864 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:39.020848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c302b831-9940-429a-abf7-13c51a6c0864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:39.089897 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 159/312] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:39.112835 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52454a-6947-4c0c-9527-4574d3bd5c39 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:39.134249 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:39.229470 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52454a-6947-4c0c-9527-4574d3bd5c39 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:39.266166 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c81f8-dd0b-483b-a9cc-ee8d57b25e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:39.267096 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c81f8-dd0b-483b-a9cc-ee8d57b25e20 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:39.268122 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c81f8-dd0b-483b-a9cc-ee8d57b25e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:39.268815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c81f8-dd0b-483b-a9cc-ee8d57b25e20 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:39.280161 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c81f8-dd0b-483b-a9cc-ee8d57b25e20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:39.286282 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-c43c81f8-dd0b-483b-a9cc-ee8d57b25e20 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 04:08:39.286282 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:39.286282 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:39.286282 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:39.286282 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:39.286282 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:39.286282 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:39.286282 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:39.286282 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:39.286282 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:39.286282 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:39.286282 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:39.286282 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:39.286282 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:39.286282 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:39.286282 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:39.287150 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:39.287150 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:39.287150 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:39.287150 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:39.287150 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:39.287150 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:39.287150 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:39.287150 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:39.287150 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:39.287150 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:39.287150 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:39.287150 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:39.287150 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:39.287150 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:39.287150 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:39.287150 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:39.287948 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:39.287948 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:39.287948 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:39.287948 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:39.287948 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:39.287948 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:39.287948 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:39.287948 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:39.287948 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:39.287948 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:39.287948 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 04:08:39.287948 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:08:39.290202 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 154/313] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:39.297866 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302b831-9940-429a-abf7-13c51a6c0864 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:39.319086 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:39.403889 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302b831-9940-429a-abf7-13c51a6c0864 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:39.438519 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2160b-4bfe-4653-80a0-3979d955189f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:39.439146 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2160b-4bfe-4653-80a0-3979d955189f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:39.439515 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2160b-4bfe-4653-80a0-3979d955189f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:39.439876 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2160b-4bfe-4653-80a0-3979d955189f None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:39.440809 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e2160b-4bfe-4653-80a0-3979d955189f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:39.441123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e2160b-4bfe-4653-80a0-3979d955189f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:39.442534 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e2160b-4bfe-4653-80a0-3979d955189f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:39.451251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 160/314] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 27 04:08:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:39.459075 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c81f8-dd0b-483b-a9cc-ee8d57b25e20 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:39.549574 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:39.579360 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c81f8-dd0b-483b-a9cc-ee8d57b25e20 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:39.620605 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc5c91a-02b8-410e-83e7-ece206a6d53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:39.621262 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc5c91a-02b8-410e-83e7-ece206a6d53c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:39.621543 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc5c91a-02b8-410e-83e7-ece206a6d53c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:39.621816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc5c91a-02b8-410e-83e7-ece206a6d53c None admin] PATH_INFO: `/v3/users/ef7b73952eec4ef6b1a0688fce3b4dd4` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:39.623035 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc5c91a-02b8-410e-83e7-ece206a6d53c None admin] RBAC: Authorizing `identity:get_user(user_id=ef7b73952eec4ef6b1a0688fce3b4dd4)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:39.624479 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc5c91a-02b8-410e-83e7-ece206a6d53c None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:39.626108 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 155/315] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:39 2022] GET /identity/v3/users/ef7b73952eec4ef6b1a0688fce3b4dd4 => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:39.632446 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2160b-4bfe-4653-80a0-3979d955189f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:39.716540 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:39.735909 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2160b-4bfe-4653-80a0-3979d955189f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:39.771391 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18711dec-aa82-409e-8f4d-643edb61313f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:39.772021 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18711dec-aa82-409e-8f4d-643edb61313f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:39.772306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18711dec-aa82-409e-8f4d-643edb61313f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:39.772583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18711dec-aa82-409e-8f4d-643edb61313f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:39.843340 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18711dec-aa82-409e-8f4d-643edb61313f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:39.844850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18711dec-aa82-409e-8f4d-643edb61313f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:39.929185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:39.930982 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a90c5-fcd4-4ef4-9a3d-86ad0f415082 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:39.931469 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a90c5-fcd4-4ef4-9a3d-86ad0f415082 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:39.931768 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a90c5-fcd4-4ef4-9a3d-86ad0f415082 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:39.933117 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 156/316] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:39.943296 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95368686-f348-488b-9205-36ec9318eae5 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:39.943585 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95368686-f348-488b-9205-36ec9318eae5 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:39.943876 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95368686-f348-488b-9205-36ec9318eae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:39.957499 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 161/317] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:08:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:08:39.963088 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18711dec-aa82-409e-8f4d-643edb61313f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:39.968964 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-95368686-f348-488b-9205-36ec9318eae5 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:39.970438 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-95368686-f348-488b-9205-36ec9318eae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:40.055803 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 157/318] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:40.077344 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18711dec-aa82-409e-8f4d-643edb61313f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:40.080497 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95368686-f348-488b-9205-36ec9318eae5 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:40.117932 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c83928-2d00-45f5-a6ef-a7ed7240529b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:40.118982 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c83928-2d00-45f5-a6ef-a7ed7240529b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:40.119507 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c83928-2d00-45f5-a6ef-a7ed7240529b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:40.119989 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c83928-2d00-45f5-a6ef-a7ed7240529b None admin] PATH_INFO: `/v3/projects/932a11c28fa2405793ce99d7097c63a5` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:40.121494 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c83928-2d00-45f5-a6ef-a7ed7240529b None admin] RBAC: Authorizing `identity:get_project(project_id=932a11c28fa2405793ce99d7097c63a5)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:40.123219 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c83928-2d00-45f5-a6ef-a7ed7240529b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:40.130377 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 162/319] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:39 2022] GET /identity/v3/projects/932a11c28fa2405793ce99d7097c63a5 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:08:40.131522 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c83928-2d00-45f5-a6ef-a7ed7240529b None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:40.187178 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:40.191887 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:40.227494 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95368686-f348-488b-9205-36ec9318eae5 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:40.247402 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c83928-2d00-45f5-a6ef-a7ed7240529b None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:40.273610 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2885ccc-914c-452d-9f45-11cca387e17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:40.274380 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2885ccc-914c-452d-9f45-11cca387e17f None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:40.275076 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2885ccc-914c-452d-9f45-11cca387e17f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:40.275372 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2885ccc-914c-452d-9f45-11cca387e17f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:40.276817 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2885ccc-914c-452d-9f45-11cca387e17f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:40.279107 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2885ccc-914c-452d-9f45-11cca387e17f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:40.281533 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 158/320] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:08:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:08:40.288299 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2885ccc-914c-452d-9f45-11cca387e17f None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:40.291962 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c02923-ac6e-43d2-b7cd-bf9277989c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:40.292699 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c02923-ac6e-43d2-b7cd-bf9277989c26 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:40.293685 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c02923-ac6e-43d2-b7cd-bf9277989c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:40.293685 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c02923-ac6e-43d2-b7cd-bf9277989c26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:40.294283 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c02923-ac6e-43d2-b7cd-bf9277989c26 None admin] RBAC: Adding query filter params (role.id=84fc927928374b91847a13b7fbb8ddd1, scope.project.id=932a11c28fa2405793ce99d7097c63a5, user.id=ef7b73952eec4ef6b1a0688fce3b4dd4) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:40.294663 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c02923-ac6e-43d2-b7cd-bf9277989c26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:40.296183 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c02923-ac6e-43d2-b7cd-bf9277989c26 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:40.310120 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 163/321] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:08:40 2022] GET /identity/v3/role_assignments?user.id=ef7b73952eec4ef6b1a0688fce3b4dd4&scope.project.id=932a11c28fa2405793ce99d7097c63a5&role.id=84fc927928374b91847a13b7fbb8ddd1 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:40.408951 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2885ccc-914c-452d-9f45-11cca387e17f None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:40.447949 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e5cb9-397a-4a6f-bbf4-5f6866630700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:40.448938 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e5cb9-397a-4a6f-bbf4-5f6866630700 None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:40.449408 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e5cb9-397a-4a6f-bbf4-5f6866630700 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:40.449878 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e5cb9-397a-4a6f-bbf4-5f6866630700 None admin] PATH_INFO: `/v3/groups` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:40.451269 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e5cb9-397a-4a6f-bbf4-5f6866630700 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:40.452964 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e5cb9-397a-4a6f-bbf4-5f6866630700 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:40.478146 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 159/322] 10.222.0.46 () {60 vars in 1134 bytes} [Wed Jul 27 04:08:40 2022] POST /identity/v3/groups => generated 226 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:08:40.508525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32234b-c587-4aba-a5be-565ff989cf8d None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:40.508813 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32234b-c587-4aba-a5be-565ff989cf8d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:40.509189 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32234b-c587-4aba-a5be-565ff989cf8d None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:40.511175 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 164/323] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:40.516708 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda58648-7166-4120-ad69-e78e08e55146 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:40.517034 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda58648-7166-4120-ad69-e78e08e55146 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:40.517319 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda58648-7166-4120-ad69-e78e08e55146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:40.525857 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:40.539201 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-cda58648-7166-4120-ad69-e78e08e55146 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:40.540448 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-cda58648-7166-4120-ad69-e78e08e55146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:40.625637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 160/324] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:40.647204 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b32234b-c587-4aba-a5be-565ff989cf8d None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:40.755891 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b32234b-c587-4aba-a5be-565ff989cf8d None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:40.771693 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:40.792806 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b323b-906c-4850-8580-0d66d32fca3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:40.793455 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b323b-906c-4850-8580-0d66d32fca3d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:40.793734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b323b-906c-4850-8580-0d66d32fca3d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:40.794032 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b323b-906c-4850-8580-0d66d32fca3d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:40.805539 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b323b-906c-4850-8580-0d66d32fca3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:40.810961 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-202b323b-906c-4850-8580-0d66d32fca3d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 04:08:40.810961 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:40.810961 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:40.810961 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:40.810961 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:40.810961 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:40.810961 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:40.810961 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:40.810961 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:40.810961 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:40.810961 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:40.810961 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:40.810961 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:40.810961 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:40.810961 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:40.810961 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:40.811858 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:40.811858 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:40.811858 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:40.811858 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:40.811858 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:40.811858 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:40.811858 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:40.811858 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:40.811858 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:40.811858 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:40.811858 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:40.811858 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:40.811858 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:40.811858 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:40.811858 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:40.811858 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:40.812622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:40.812622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:40.812622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:40.812622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:40.812622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:40.812622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:40.812622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:40.812622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:40.812622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:40.812622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:40.812622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 04:08:40.812622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:08:40.812622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 165/325] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:40.819456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda58648-7166-4120-ad69-e78e08e55146 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:40.934573 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda58648-7166-4120-ad69-e78e08e55146 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:40.975773 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c7ce89-bdcd-408e-95ef-cc6a3ad91079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:40.976782 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7ce89-bdcd-408e-95ef-cc6a3ad91079 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:40.977260 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7ce89-bdcd-408e-95ef-cc6a3ad91079 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:40.977740 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7ce89-bdcd-408e-95ef-cc6a3ad91079 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:40.978720 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c7ce89-bdcd-408e-95ef-cc6a3ad91079 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:40.979238 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c7ce89-bdcd-408e-95ef-cc6a3ad91079 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:40.981050 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c7ce89-bdcd-408e-95ef-cc6a3ad91079 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:40.982810 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:40.992675 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 161/326] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 27 04:08:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:40.998568 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b323b-906c-4850-8580-0d66d32fca3d None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:41.003781 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca18ac63-c9df-4061-9f34-e6609c2c75f6 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:41.004118 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca18ac63-c9df-4061-9f34-e6609c2c75f6 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:41.004627 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca18ac63-c9df-4061-9f34-e6609c2c75f6 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:41.007690 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 162/327] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:41.012443 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149d445-f1c4-46fd-9f0f-a5a4b1d9160d None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:41.012733 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149d445-f1c4-46fd-9f0f-a5a4b1d9160d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:41.013015 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149d445-f1c4-46fd-9f0f-a5a4b1d9160d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:41.035153 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-5149d445-f1c4-46fd-9f0f-a5a4b1d9160d None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:41.035804 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-5149d445-f1c4-46fd-9f0f-a5a4b1d9160d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:41.117915 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 163/328] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 04:08:41.136555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b323b-906c-4850-8580-0d66d32fca3d None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:41.142003 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149d445-f1c4-46fd-9f0f-a5a4b1d9160d None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:41.182220 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043cafdd-5305-4db2-a51e-26801299dc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:41.183448 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043cafdd-5305-4db2-a51e-26801299dc0d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:41.184162 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043cafdd-5305-4db2-a51e-26801299dc0d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:41.184615 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043cafdd-5305-4db2-a51e-26801299dc0d None admin] PATH_INFO: `/v3/users/7af85767ac7a490e8c36341b5cc9d5ed` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:41.186654 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043cafdd-5305-4db2-a51e-26801299dc0d None admin] RBAC: Authorizing `identity:get_user(user_id=7af85767ac7a490e8c36341b5cc9d5ed)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:41.188916 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043cafdd-5305-4db2-a51e-26801299dc0d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:41.192591 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 166/329] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:40 2022] GET /identity/v3/users/7af85767ac7a490e8c36341b5cc9d5ed => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:41.201214 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043cafdd-5305-4db2-a51e-26801299dc0d None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:41.256232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:41.276873 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149d445-f1c4-46fd-9f0f-a5a4b1d9160d None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:41.307329 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:41.319923 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c78ffa-cec3-4e7d-852f-2d485a19121a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:41.321055 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c78ffa-cec3-4e7d-852f-2d485a19121a None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:41.321539 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c78ffa-cec3-4e7d-852f-2d485a19121a None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:41.322045 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c78ffa-cec3-4e7d-852f-2d485a19121a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:41.330740 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043cafdd-5305-4db2-a51e-26801299dc0d None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:41.336599 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c78ffa-cec3-4e7d-852f-2d485a19121a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:41.342170 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-99c78ffa-cec3-4e7d-852f-2d485a19121a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:41.342170 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:41.342170 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:41.342170 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:41.342170 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:41.342170 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:41.342170 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:41.342170 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:41.342170 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:41.342170 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:41.342170 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:41.342170 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:41.342170 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:41.342170 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:41.342170 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:41.342170 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:41.343022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:41.343022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:41.343022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:41.343022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:41.343022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:41.343022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:41.343022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:41.343022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:41.343022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:41.343022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:41.343022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:41.343022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:41.343022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:41.343022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:41.343022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:41.343022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:41.343810 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:41.343810 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:41.343810 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:41.343810 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:41.343810 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:41.343810 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:41.343810 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:41.343810 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:41.343810 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:41.343810 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:41.343810 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:41.343810 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:08:41.347462 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 164/330] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:41.353501 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c78ffa-cec3-4e7d-852f-2d485a19121a None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:41.369561 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064acc6-0725-4402-8519-6ae1301ae62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:41.370328 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064acc6-0725-4402-8519-6ae1301ae62f None admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:41.370609 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064acc6-0725-4402-8519-6ae1301ae62f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:41.370888 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064acc6-0725-4402-8519-6ae1301ae62f None admin] PATH_INFO: `/v3/system/users/7af85767ac7a490e8c36341b5cc9d5ed/roles/84fc927928374b91847a13b7fbb8ddd1` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:41.372549 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7064acc6-0725-4402-8519-6ae1301ae62f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7af85767ac7a490e8c36341b5cc9d5ed, role_id=84fc927928374b91847a13b7fbb8ddd1)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:41.374731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7064acc6-0725-4402-8519-6ae1301ae62f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:41.385107 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 167/331] 10.222.0.46 () {58 vars in 1334 bytes} [Wed Jul 27 04:08:41 2022] PUT /identity/v3/system/users/7af85767ac7a490e8c36341b5cc9d5ed/roles/84fc927928374b91847a13b7fbb8ddd1 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:08:41.486004 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c78ffa-cec3-4e7d-852f-2d485a19121a None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:41.527208 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f977b2-7882-41fe-a83a-2c3fcb1316a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:41.528189 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f977b2-7882-41fe-a83a-2c3fcb1316a9 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:41.528686 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f977b2-7882-41fe-a83a-2c3fcb1316a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:41.529152 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f977b2-7882-41fe-a83a-2c3fcb1316a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:41.530119 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f977b2-7882-41fe-a83a-2c3fcb1316a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:41.530658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f977b2-7882-41fe-a83a-2c3fcb1316a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:41.532243 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f977b2-7882-41fe-a83a-2c3fcb1316a9 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:41.543203 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 165/332] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 27 04:08:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:41.549861 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064acc6-0725-4402-8519-6ae1301ae62f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:41.680613 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064acc6-0725-4402-8519-6ae1301ae62f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:41.698473 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:41.719237 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84060827-f28d-4947-b9ba-883699bdd0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:41.719893 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84060827-f28d-4947-b9ba-883699bdd0c1 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:41.720176 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84060827-f28d-4947-b9ba-883699bdd0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:41.720449 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84060827-f28d-4947-b9ba-883699bdd0c1 None admin] PATH_INFO: `/v3/users/60577426a96040768a73e7119552bb6d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:41.722419 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84060827-f28d-4947-b9ba-883699bdd0c1 None admin] RBAC: Authorizing `identity:get_user(user_id=60577426a96040768a73e7119552bb6d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:41.724241 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84060827-f28d-4947-b9ba-883699bdd0c1 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:41.726571 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 168/333] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:41 2022] GET /identity/v3/users/60577426a96040768a73e7119552bb6d => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:41.732193 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f977b2-7882-41fe-a83a-2c3fcb1316a9 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:41.807606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:41.871574 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f977b2-7882-41fe-a83a-2c3fcb1316a9 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:41.917705 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4b5a1-896b-402f-95a7-a66088f5c3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:41.918355 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4b5a1-896b-402f-95a7-a66088f5c3d8 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:41.918634 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4b5a1-896b-402f-95a7-a66088f5c3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:41.918898 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4b5a1-896b-402f-95a7-a66088f5c3d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:41.919695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c4b5a1-896b-402f-95a7-a66088f5c3d8 None admin] RBAC: Adding query filter params (role.id=7b6f2ade82434ee79175385de17049eb, user.id=60577426a96040768a73e7119552bb6d, scope.system=all) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:41.919997 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c4b5a1-896b-402f-95a7-a66088f5c3d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:41.921441 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c4b5a1-896b-402f-95a7-a66088f5c3d8 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:41.936044 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 166/334] 10.222.0.46 () {56 vars in 1308 bytes} [Wed Jul 27 04:08:41 2022] GET /identity/v3/role_assignments?user.id=60577426a96040768a73e7119552bb6d&scope.system=all&role.id=7b6f2ade82434ee79175385de17049eb => generated 510 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:08:43.139999 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88901f0-7469-4a63-b8b9-0667fbe93111 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:43.140501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88901f0-7469-4a63-b8b9-0667fbe93111 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:43.140576 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88901f0-7469-4a63-b8b9-0667fbe93111 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:43.142168 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 169/335] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:43.147995 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d97e7-b113-4a74-9305-17fd20427360 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:43.148410 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d97e7-b113-4a74-9305-17fd20427360 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:43.148583 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d97e7-b113-4a74-9305-17fd20427360 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:43.150130 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 167/336] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:43.151645 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76611756-2d42-411d-b563-b1a9c84bb0ea None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:43.151925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76611756-2d42-411d-b563-b1a9c84bb0ea None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:43.152199 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76611756-2d42-411d-b563-b1a9c84bb0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:43.171995 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f884b14-5e5b-454b-ad26-7fea7ce166fe None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:43.172314 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f884b14-5e5b-454b-ad26-7fea7ce166fe None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:43.172556 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f884b14-5e5b-454b-ad26-7fea7ce166fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:43.174891 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-76611756-2d42-411d-b563-b1a9c84bb0ea None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:43.176118 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-76611756-2d42-411d-b563-b1a9c84bb0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:43.199061 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-2f884b14-5e5b-454b-ad26-7fea7ce166fe None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:43.200273 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2f884b14-5e5b-454b-ad26-7fea7ce166fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:43.318738 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 168/337] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 04:08:43.330202 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 170/338] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:43.353150 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f884b14-5e5b-454b-ad26-7fea7ce166fe None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:43.364807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76611756-2d42-411d-b563-b1a9c84bb0ea None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:43.432369 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:43.527506 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76611756-2d42-411d-b563-b1a9c84bb0ea None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:43.530039 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:43.537042 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f884b14-5e5b-454b-ad26-7fea7ce166fe None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:43.568653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbdc8ed-c0ac-4493-90f2-37f38467c318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:43.569416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbdc8ed-c0ac-4493-90f2-37f38467c318 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:43.569708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbdc8ed-c0ac-4493-90f2-37f38467c318 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:43.570010 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbdc8ed-c0ac-4493-90f2-37f38467c318 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:43.581206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbdc8ed-c0ac-4493-90f2-37f38467c318 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:43.588025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-0cbdc8ed-c0ac-4493-90f2-37f38467c318 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 04:08:43.588025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:43.588025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:43.588025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:43.588025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:43.588025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:43.588025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:43.588025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:43.588025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:43.588025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:43.588025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:43.588025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:43.588025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:43.588025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:43.588025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:43.588025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:43.588885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:43.588885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:43.588885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:43.588885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:43.588885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:43.588885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:43.588885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:43.588885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:43.588885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:43.588885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:43.588885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:43.588885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:43.588885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:43.588885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:43.588885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:43.588885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:43.589653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:43.589653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:43.589653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:43.589653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:43.589653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:43.589653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:43.589653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:43.589653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:43.589653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:43.589653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:43.589653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 04:08:43.589653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:08:43.589653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 171/339] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 27 04:08:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:08:43.597210 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbdc8ed-c0ac-4493-90f2-37f38467c318 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:43.598210 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f770b45-90fd-4d7a-a2ca-08bdd6c45fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:43.598870 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f770b45-90fd-4d7a-a2ca-08bdd6c45fae None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:43.599150 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f770b45-90fd-4d7a-a2ca-08bdd6c45fae None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:43.599424 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f770b45-90fd-4d7a-a2ca-08bdd6c45fae None admin] PATH_INFO: `/v3/roles/member` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:43.628988 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f770b45-90fd-4d7a-a2ca-08bdd6c45fae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:43.633874 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-0f770b45-90fd-4d7a-a2ca-08bdd6c45fae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:43.633874 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:43.633874 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:43.633874 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:43.633874 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:43.633874 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:43.633874 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:43.633874 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:43.633874 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:43.633874 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:43.633874 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:43.633874 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:43.633874 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:43.633874 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:43.633874 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:43.633874 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:43.635967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:43.635967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:43.635967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:43.635967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:43.635967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:43.635967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:43.635967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:43.635967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:43.635967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:43.635967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:43.635967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:43.635967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:43.635967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:43.635967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:43.635967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:43.635967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:43.637870 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:43.637870 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:43.637870 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:43.637870 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:43.637870 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:43.637870 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:43.637870 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:43.637870 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:43.637870 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:43.637870 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:43.637870 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:43.637870 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:08:43.637870 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 169/340] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:43.642181 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f770b45-90fd-4d7a-a2ca-08bdd6c45fae None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:43.712822 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbdc8ed-c0ac-4493-90f2-37f38467c318 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:43.750884 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90978fac-e7f2-4cbb-91b6-d4991baf0524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:43.751523 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90978fac-e7f2-4cbb-91b6-d4991baf0524 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:43.751939 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90978fac-e7f2-4cbb-91b6-d4991baf0524 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:43.752364 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90978fac-e7f2-4cbb-91b6-d4991baf0524 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:43.753226 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90978fac-e7f2-4cbb-91b6-d4991baf0524 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:43.753539 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90978fac-e7f2-4cbb-91b6-d4991baf0524 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:43.754982 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90978fac-e7f2-4cbb-91b6-d4991baf0524 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:43.763986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 172/341] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:08:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:43.770979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90978fac-e7f2-4cbb-91b6-d4991baf0524 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:43.821670 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:43.859372 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:43.868426 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f770b45-90fd-4d7a-a2ca-08bdd6c45fae None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:43.926850 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d66c69-1660-4488-baa0-f4e177c1f254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:43.927490 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d66c69-1660-4488-baa0-f4e177c1f254 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:43.927775 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d66c69-1660-4488-baa0-f4e177c1f254 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:43.928054 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d66c69-1660-4488-baa0-f4e177c1f254 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:43.928793 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d66c69-1660-4488-baa0-f4e177c1f254 None admin] RBAC: Adding query filter params (name=member) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:43.929102 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d66c69-1660-4488-baa0-f4e177c1f254 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:43.930547 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d66c69-1660-4488-baa0-f4e177c1f254 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:43.947571 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 170/342] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 27 04:08:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:43.957692 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d66c69-1660-4488-baa0-f4e177c1f254 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:43.992226 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90978fac-e7f2-4cbb-91b6-d4991baf0524 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:44.042862 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd883969-9153-4444-a81f-2a81c05432e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:44.043633 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd883969-9153-4444-a81f-2a81c05432e6 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:44.043925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd883969-9153-4444-a81f-2a81c05432e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:44.044197 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd883969-9153-4444-a81f-2a81c05432e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:44.137467 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d66c69-1660-4488-baa0-f4e177c1f254 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:44.199087 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd883969-9153-4444-a81f-2a81c05432e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:44.200553 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd883969-9153-4444-a81f-2a81c05432e6 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:44.209442 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61067ea3-0542-41ec-8aa9-ef55a0764718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:44.210836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61067ea3-0542-41ec-8aa9-ef55a0764718 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:44.211322 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61067ea3-0542-41ec-8aa9-ef55a0764718 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:44.211799 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61067ea3-0542-41ec-8aa9-ef55a0764718 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:44.255417 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:44.279704 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:44.365873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 173/343] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:08:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:08:44.373066 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd883969-9153-4444-a81f-2a81c05432e6 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:44.375895 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61067ea3-0542-41ec-8aa9-ef55a0764718 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:44.377907 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61067ea3-0542-41ec-8aa9-ef55a0764718 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:44.544428 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd883969-9153-4444-a81f-2a81c05432e6 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:44.586447 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 171/344] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:08:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 629 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 04:08:44.590440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:44.591873 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61067ea3-0542-41ec-8aa9-ef55a0764718 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:44.600064 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146896b8-824c-4ff7-a244-6a42923547e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:44.600737 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146896b8-824c-4ff7-a244-6a42923547e3 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:44.601021 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146896b8-824c-4ff7-a244-6a42923547e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:44.601462 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146896b8-824c-4ff7-a244-6a42923547e3 None admin] PATH_INFO: `/v3/projects/85168a036bf94350836ae48489878ed3` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:44.611964 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146896b8-824c-4ff7-a244-6a42923547e3 None admin] RBAC: Authorizing `identity:get_project(project_id=85168a036bf94350836ae48489878ed3)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:44.611964 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146896b8-824c-4ff7-a244-6a42923547e3 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:44.611964 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 174/345] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:44 2022] GET /identity/v3/projects/85168a036bf94350836ae48489878ed3 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:44.616917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146896b8-824c-4ff7-a244-6a42923547e3 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:44.706015 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:44.787333 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61067ea3-0542-41ec-8aa9-ef55a0764718 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:44.827788 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146896b8-824c-4ff7-a244-6a42923547e3 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:44.840395 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010d6c6-db3f-405e-8216-79c54b042336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:44.841050 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010d6c6-db3f-405e-8216-79c54b042336 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:44.841328 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010d6c6-db3f-405e-8216-79c54b042336 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:44.841599 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010d6c6-db3f-405e-8216-79c54b042336 None admin] PATH_INFO: `/v3/projects/85168a036bf94350836ae48489878ed3` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:44.842804 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010d6c6-db3f-405e-8216-79c54b042336 None admin] RBAC: Authorizing `identity:get_project(project_id=85168a036bf94350836ae48489878ed3)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:44.844250 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010d6c6-db3f-405e-8216-79c54b042336 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:44.846960 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 172/346] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:44 2022] GET /identity/v3/projects/85168a036bf94350836ae48489878ed3 => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:08:44.848251 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9ba4d-c146-4483-944c-2116830601d7 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:44.848530 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9ba4d-c146-4483-944c-2116830601d7 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:44.848805 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9ba4d-c146-4483-944c-2116830601d7 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:44.850315 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 173/347] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:44.852997 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9ba4d-c146-4483-944c-2116830601d7 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:44.906686 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce2a1db-3928-40c3-b312-9b0350aa049e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:44.907364 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2a1db-3928-40c3-b312-9b0350aa049e None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:44.907642 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2a1db-3928-40c3-b312-9b0350aa049e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:44.908470 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2a1db-3928-40c3-b312-9b0350aa049e None admin] PATH_INFO: `/v3/groups/b4aab270fc5642f2b77d88507a8b6c2b` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:44.918788 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce2a1db-3928-40c3-b312-9b0350aa049e None admin] RBAC: Authorizing `identity:get_group(group_id=b4aab270fc5642f2b77d88507a8b6c2b)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:44.919637 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce2a1db-3928-40c3-b312-9b0350aa049e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:44.921633 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 175/348] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Jul 27 04:08:44 2022] GET /identity/v3/groups/b4aab270fc5642f2b77d88507a8b6c2b => generated 226 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:08:44.922839 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ea7e0-340a-42e6-86e9-f6b33ebbeaa3 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:44.923123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ea7e0-340a-42e6-86e9-f6b33ebbeaa3 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:44.923419 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ea7e0-340a-42e6-86e9-f6b33ebbeaa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:44.950702 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-f89ea7e0-340a-42e6-86e9-f6b33ebbeaa3 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:44.951913 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f89ea7e0-340a-42e6-86e9-f6b33ebbeaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:44.980420 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9ba4d-c146-4483-944c-2116830601d7 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:45.016516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:45.028738 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3dd3f4-30c4-46d4-8691-e28bfcd42578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:45.029393 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3dd3f4-30c4-46d4-8691-e28bfcd42578 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:45.029675 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3dd3f4-30c4-46d4-8691-e28bfcd42578 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:45.030005 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3dd3f4-30c4-46d4-8691-e28bfcd42578 None admin] PATH_INFO: `/v3/groups/b4aab270fc5642f2b77d88507a8b6c2b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:45.030905 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:45.032748 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3dd3f4-30c4-46d4-8691-e28bfcd42578 None admin] RBAC: Authorizing `identity:get_group(group_id=b4aab270fc5642f2b77d88507a8b6c2b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:45.034722 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3dd3f4-30c4-46d4-8691-e28bfcd42578 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:45.038011 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 174/349] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Jul 27 04:08:44 2022] GET /identity/v3/groups/b4aab270fc5642f2b77d88507a8b6c2b => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:45.039497 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5fc3b-3db6-4db5-a888-787359b7bae5 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:45.039776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5fc3b-3db6-4db5-a888-787359b7bae5 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:45.040047 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5fc3b-3db6-4db5-a888-787359b7bae5 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:45.042057 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 175/350] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:45.043025 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e5fc3b-3db6-4db5-a888-787359b7bae5 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:45.048973 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 176/351] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:45.050060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ea7e0-340a-42e6-86e9-f6b33ebbeaa3 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:45.183469 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ea7e0-340a-42e6-86e9-f6b33ebbeaa3 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:45.201082 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e5fc3b-3db6-4db5-a888-787359b7bae5 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:45.237060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac03f03-dc36-4fe0-b789-a6d4347ecf2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:45.237794 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac03f03-dc36-4fe0-b789-a6d4347ecf2d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:45.238065 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac03f03-dc36-4fe0-b789-a6d4347ecf2d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:45.238340 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac03f03-dc36-4fe0-b789-a6d4347ecf2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:45.239316 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac03f03-dc36-4fe0-b789-a6d4347ecf2d None admin] RBAC: Adding query filter params (group.id=b4aab270fc5642f2b77d88507a8b6c2b, role.id=7b6f2ade82434ee79175385de17049eb, scope.project.id=85168a036bf94350836ae48489878ed3) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:45.239630 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac03f03-dc36-4fe0-b789-a6d4347ecf2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:45.241511 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac03f03-dc36-4fe0-b789-a6d4347ecf2d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:45.249781 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015840e-68d9-45e2-a2dc-b373739e3c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:45.250486 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 177/352] 10.222.0.46 () {56 vars in 1376 bytes} [Wed Jul 27 04:08:45 2022] GET /identity/v3/role_assignments?group.id=b4aab270fc5642f2b77d88507a8b6c2b&scope.project.id=85168a036bf94350836ae48489878ed3&role.id=7b6f2ade82434ee79175385de17049eb => generated 260 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:45.251397 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015840e-68d9-45e2-a2dc-b373739e3c98 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:45.251775 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0254894-aba0-481b-be9b-e63f9bb79095 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:45.252124 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0254894-aba0-481b-be9b-e63f9bb79095 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:45.252400 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0254894-aba0-481b-be9b-e63f9bb79095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:45.252917 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015840e-68d9-45e2-a2dc-b373739e3c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:45.253770 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015840e-68d9-45e2-a2dc-b373739e3c98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:45.255374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7015840e-68d9-45e2-a2dc-b373739e3c98 None admin] RBAC: Adding query filter params (group.id=b4aab270fc5642f2b77d88507a8b6c2b, role.id=cdfc87279ea24b65bae3961035133725, scope.project.id=85168a036bf94350836ae48489878ed3) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:45.255937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7015840e-68d9-45e2-a2dc-b373739e3c98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:45.257678 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7015840e-68d9-45e2-a2dc-b373739e3c98 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:45.269604 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 176/353] 10.222.0.46 () {56 vars in 1376 bytes} [Wed Jul 27 04:08:45 2022] GET /identity/v3/role_assignments?group.id=b4aab270fc5642f2b77d88507a8b6c2b&scope.project.id=85168a036bf94350836ae48489878ed3&role.id=cdfc87279ea24b65bae3961035133725 => generated 260 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:45.272393 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015840e-68d9-45e2-a2dc-b373739e3c98 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:45.288196 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-c0254894-aba0-481b-be9b-e63f9bb79095 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:45.289897 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c0254894-aba0-481b-be9b-e63f9bb79095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:45.383383 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:45.384739 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:45.418056 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 178/354] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:45.424787 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ccebd-2fbd-40fa-a174-842a338d5b71 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:45.427096 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ccebd-2fbd-40fa-a174-842a338d5b71 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:45.429139 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ccebd-2fbd-40fa-a174-842a338d5b71 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:45.435721 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 179/355] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:45 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:45.440526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22737f3e-3ad5-442f-a9bd-9ab68af00f21 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:45.442971 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22737f3e-3ad5-442f-a9bd-9ab68af00f21 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:45.443468 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22737f3e-3ad5-442f-a9bd-9ab68af00f21 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:45.445456 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 180/356] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:45 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:45.447155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb2a0a3-a412-40bd-9c28-feca232304b7 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:45.449122 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb2a0a3-a412-40bd-9c28-feca232304b7 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:45.450384 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb2a0a3-a412-40bd-9c28-feca232304b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:45.475337 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015840e-68d9-45e2-a2dc-b373739e3c98 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:45.480209 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-3fb2a0a3-a412-40bd-9c28-feca232304b7 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:45.481402 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-3fb2a0a3-a412-40bd-9c28-feca232304b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:45.521756 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e62da-2ba3-4a21-8555-078d3d04b328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:45.522817 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e62da-2ba3-4a21-8555-078d3d04b328 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:45.523302 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e62da-2ba3-4a21-8555-078d3d04b328 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:45.523815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e62da-2ba3-4a21-8555-078d3d04b328 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:45.538073 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3e62da-2ba3-4a21-8555-078d3d04b328 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:45.546834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-5e3e62da-2ba3-4a21-8555-078d3d04b328 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 04:08:45.546834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:45.546834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:45.546834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:45.546834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:45.546834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:45.546834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:45.546834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:45.546834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:45.546834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:45.546834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:45.546834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:45.546834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:45.546834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:45.546834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:45.546834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:45.547779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:45.547779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:45.547779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:45.547779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:45.547779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:45.547779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:45.547779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:45.547779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:45.547779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:45.547779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:45.547779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:45.547779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:45.547779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:45.547779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:45.547779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:45.547779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:45.549158 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:45.549158 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:45.549158 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:45.549158 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:45.549158 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:45.549158 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:45.549158 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:45.549158 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:45.549158 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:45.549158 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:45.549158 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 04:08:45.549158 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:08:45.552764 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 177/357] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:45.554598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547914d-b104-4a9e-bbfa-6b243a80e086 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:45.555207 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547914d-b104-4a9e-bbfa-6b243a80e086 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:45.555667 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547914d-b104-4a9e-bbfa-6b243a80e086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:45.572725 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 181/358] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:45.572725 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb2a0a3-a412-40bd-9c28-feca232304b7 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:45.589579 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-1547914d-b104-4a9e-bbfa-6b243a80e086 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:45.591505 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-1547914d-b104-4a9e-bbfa-6b243a80e086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:45.683323 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 178/359] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:45.684386 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1547914d-b104-4a9e-bbfa-6b243a80e086 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:45.709525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb2a0a3-a412-40bd-9c28-feca232304b7 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:45.729315 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:45.759153 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ecfe9-81ad-43a3-9e4d-53d34aeae409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:45.760058 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ecfe9-81ad-43a3-9e4d-53d34aeae409 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:45.760250 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ecfe9-81ad-43a3-9e4d-53d34aeae409 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:45.760539 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ecfe9-81ad-43a3-9e4d-53d34aeae409 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:45.766399 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:45.778461 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6ecfe9-81ad-43a3-9e4d-53d34aeae409 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:45.785079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-ed6ecfe9-81ad-43a3-9e4d-53d34aeae409 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 04:08:45.785079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:45.785079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:45.785079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:45.785079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:45.785079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:45.785079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:45.785079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:45.785079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:45.785079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:45.785079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:45.785079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:45.785079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:45.785079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:45.785079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:45.785079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:45.786018 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:45.786018 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:45.786018 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:45.786018 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:45.786018 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:45.786018 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:45.786018 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:45.786018 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:45.786018 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:45.786018 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:45.786018 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:45.786018 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:45.786018 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:45.786018 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:45.786018 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:45.786018 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:45.786836 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:45.786836 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:45.786836 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:45.786836 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:45.786836 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:45.786836 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:45.786836 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:45.786836 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:45.786836 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:45.786836 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:45.786836 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 04:08:45.786836 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:08:45.789775 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 182/360] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 27 04:08:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:45.791229 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ecfe9-81ad-43a3-9e4d-53d34aeae409 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:45.808357 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1547914d-b104-4a9e-bbfa-6b243a80e086 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:45.856505 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0462a1-6c79-4e91-be31-61e096ecb548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:45.857145 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0462a1-6c79-4e91-be31-61e096ecb548 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:45.857429 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0462a1-6c79-4e91-be31-61e096ecb548 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:45.857708 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0462a1-6c79-4e91-be31-61e096ecb548 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:45.858484 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0462a1-6c79-4e91-be31-61e096ecb548 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:45.858790 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0462a1-6c79-4e91-be31-61e096ecb548 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:45.860174 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0462a1-6c79-4e91-be31-61e096ecb548 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:45.870611 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 179/361] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 27 04:08:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:45.871455 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0462a1-6c79-4e91-be31-61e096ecb548 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:45.923441 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ecfe9-81ad-43a3-9e4d-53d34aeae409 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:45.976665 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de673cd5-0e18-4b06-98f1-17c132ae9586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:45.977247 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de673cd5-0e18-4b06-98f1-17c132ae9586 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:45.977529 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de673cd5-0e18-4b06-98f1-17c132ae9586 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:45.977807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de673cd5-0e18-4b06-98f1-17c132ae9586 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:45.994993 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de673cd5-0e18-4b06-98f1-17c132ae9586 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:46.003043 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-de673cd5-0e18-4b06-98f1-17c132ae9586 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:46.003043 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:46.003043 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:46.003043 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:46.003043 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:46.003043 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:46.003043 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:46.003043 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:46.003043 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:46.003043 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:46.003043 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:46.003043 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:46.003043 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:46.003043 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:46.003043 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:46.003043 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:46.003953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:46.003953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:46.003953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:46.003953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:46.003953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:46.003953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:46.003953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:46.003953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:46.003953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:46.003953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:46.003953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:46.003953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:46.003953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:46.003953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:46.003953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:46.003953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:46.004732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:46.004732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:46.004732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:46.004732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:46.004732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:46.004732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:46.004732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:46.004732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:46.004732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:46.004732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:46.004732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:46.004732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:08:46.004732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 183/362] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:46.004732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de673cd5-0e18-4b06-98f1-17c132ae9586 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:46.024845 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0462a1-6c79-4e91-be31-61e096ecb548 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:46.037178 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:46.043252 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:46.066485 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1249a3-2f68-464a-9778-96def1083d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:46.067165 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1249a3-2f68-464a-9778-96def1083d5a None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:46.067444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1249a3-2f68-464a-9778-96def1083d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:46.067717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1249a3-2f68-464a-9778-96def1083d5a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:46.078750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1249a3-2f68-464a-9778-96def1083d5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:46.087142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-ab1249a3-2f68-464a-9778-96def1083d5a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:08:46.087142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:46.087142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:46.087142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:46.087142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:46.087142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:46.087142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:46.087142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:46.087142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:46.087142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:46.087142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:46.087142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:46.087142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:46.087142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:46.087142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:46.087142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:46.088054 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:46.088054 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:46.088054 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:46.088054 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:46.088054 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:46.088054 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:46.088054 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:46.088054 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:46.088054 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:46.088054 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:46.088054 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:46.088054 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:46.088054 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:46.088054 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:46.088054 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:46.088054 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:46.089252 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:46.089252 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:46.089252 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:46.089252 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:46.089252 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:46.089252 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:46.089252 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:46.089252 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:46.089252 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:46.089252 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:46.089252 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:08:46.089252 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:08:46.089252 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 180/363] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:08:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:46.089917 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1249a3-2f68-464a-9778-96def1083d5a None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:46.121549 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de673cd5-0e18-4b06-98f1-17c132ae9586 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:46.156915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d249d5f-2057-4496-aa27-300d59056ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:46.157558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d249d5f-2057-4496-aa27-300d59056ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:46.157841 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d249d5f-2057-4496-aa27-300d59056ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:46.158892 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d249d5f-2057-4496-aa27-300d59056ed4 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:46.159640 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d249d5f-2057-4496-aa27-300d59056ed4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:46.159949 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d249d5f-2057-4496-aa27-300d59056ed4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:46.161337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d249d5f-2057-4496-aa27-300d59056ed4 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:46.171026 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 184/364] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:08:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:46.172238 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d249d5f-2057-4496-aa27-300d59056ed4 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:46.208651 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1249a3-2f68-464a-9778-96def1083d5a None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:46.246202 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53529219-9f10-4da2-b78c-fcb57a1187bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:46.246785 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53529219-9f10-4da2-b78c-fcb57a1187bb None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:46.247064 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53529219-9f10-4da2-b78c-fcb57a1187bb None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:46.247337 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53529219-9f10-4da2-b78c-fcb57a1187bb None admin] PATH_INFO: `/v3/users/7af85767ac7a490e8c36341b5cc9d5ed` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:46.249350 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53529219-9f10-4da2-b78c-fcb57a1187bb None admin] RBAC: Authorizing `identity:get_user(user_id=7af85767ac7a490e8c36341b5cc9d5ed)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:46.250814 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53529219-9f10-4da2-b78c-fcb57a1187bb None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:46.252827 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 181/365] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:08:46 2022] GET /identity/v3/users/7af85767ac7a490e8c36341b5cc9d5ed => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:46.253658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53529219-9f10-4da2-b78c-fcb57a1187bb None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:46.278381 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:46.284824 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d249d5f-2057-4496-aa27-300d59056ed4 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:46.301514 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:46.321161 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db032d-8e4d-4d85-a9b0-62c61cbc8ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:46.321800 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db032d-8e4d-4d85-a9b0-62c61cbc8ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:46.322113 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db032d-8e4d-4d85-a9b0-62c61cbc8ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:46.322398 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db032d-8e4d-4d85-a9b0-62c61cbc8ec6 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:46.323128 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db032d-8e4d-4d85-a9b0-62c61cbc8ec6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:46.323441 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db032d-8e4d-4d85-a9b0-62c61cbc8ec6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:46.324960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db032d-8e4d-4d85-a9b0-62c61cbc8ec6 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:46.335161 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 185/366] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 27 04:08:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:46.336008 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db032d-8e4d-4d85-a9b0-62c61cbc8ec6 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:46.365983 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53529219-9f10-4da2-b78c-fcb57a1187bb None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:46.403172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a5ca0-c1f4-48bb-92e2-2c9f1f96deed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:46.403798 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a5ca0-c1f4-48bb-92e2-2c9f1f96deed None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:46.404076 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a5ca0-c1f4-48bb-92e2-2c9f1f96deed None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:46.404347 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a5ca0-c1f4-48bb-92e2-2c9f1f96deed None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:46.405068 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a5ca0-c1f4-48bb-92e2-2c9f1f96deed None admin] RBAC: Adding query filter params (name=admin) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:46.405373 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a5ca0-c1f4-48bb-92e2-2c9f1f96deed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:46.406800 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a5ca0-c1f4-48bb-92e2-2c9f1f96deed None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:46.422282 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 182/367] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:46.423198 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a5ca0-c1f4-48bb-92e2-2c9f1f96deed None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:46.444659 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db032d-8e4d-4d85-a9b0-62c61cbc8ec6 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:46.483617 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e2087-e091-4709-8458-708a003b19e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:46.484254 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e2087-e091-4709-8458-708a003b19e2 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:46.484544 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e2087-e091-4709-8458-708a003b19e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:46.484824 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e2087-e091-4709-8458-708a003b19e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:46.514434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:46.544645 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:46.549171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a5ca0-c1f4-48bb-92e2-2c9f1f96deed None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:46.563632 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e2087-e091-4709-8458-708a003b19e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:46.565157 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e2087-e091-4709-8458-708a003b19e2 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:46.590583 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b0922-f854-4762-9873-f9c55ceb3964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:46.591475 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b0922-f854-4762-9873-f9c55ceb3964 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:46.591901 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b0922-f854-4762-9873-f9c55ceb3964 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:46.592338 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b0922-f854-4762-9873-f9c55ceb3964 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:46.593282 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b0922-f854-4762-9873-f9c55ceb3964 None admin] RBAC: Adding query filter params (role.id=84fc927928374b91847a13b7fbb8ddd1, user.id=7af85767ac7a490e8c36341b5cc9d5ed, scope.system=all) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:46.593734 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b0922-f854-4762-9873-f9c55ceb3964 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:46.596008 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b0922-f854-4762-9873-f9c55ceb3964 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:46.610379 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 183/368] 10.222.0.46 () {56 vars in 1308 bytes} [Wed Jul 27 04:08:46 2022] GET /identity/v3/role_assignments?user.id=7af85767ac7a490e8c36341b5cc9d5ed&scope.system=all&role.id=84fc927928374b91847a13b7fbb8ddd1 => generated 510 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:46.618025 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b0922-f854-4762-9873-f9c55ceb3964 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:46.695679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 186/369] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:08:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:08:46.696625 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e2087-e091-4709-8458-708a003b19e2 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:46.766498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:46.816793 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e2087-e091-4709-8458-708a003b19e2 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:46.856480 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ba8b6-5385-4f8e-b605-c827a82be99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:46.857127 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ba8b6-5385-4f8e-b605-c827a82be99c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:46.857412 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ba8b6-5385-4f8e-b605-c827a82be99c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:46.857827 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ba8b6-5385-4f8e-b605-c827a82be99c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:46.898299 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b0922-f854-4762-9873-f9c55ceb3964 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:46.949404 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80638d-f9ff-49e1-b15d-be954d162f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:46.950088 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80638d-f9ff-49e1-b15d-be954d162f88 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:46.950377 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80638d-f9ff-49e1-b15d-be954d162f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:46.950655 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80638d-f9ff-49e1-b15d-be954d162f88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:46.957956 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ba8b6-5385-4f8e-b605-c827a82be99c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:46.959733 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ba8b6-5385-4f8e-b605-c827a82be99c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:46.987449 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:47.041828 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d80638d-f9ff-49e1-b15d-be954d162f88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:47.042710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d80638d-f9ff-49e1-b15d-be954d162f88 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:47.050421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:47.121370 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 187/370] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:08:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:08:47.122762 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ba8b6-5385-4f8e-b605-c827a82be99c None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:47.151585 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 184/371] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:08:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:08:47.152442 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80638d-f9ff-49e1-b15d-be954d162f88 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:47.214545 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:47.238498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ba8b6-5385-4f8e-b605-c827a82be99c None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:47.267264 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80638d-f9ff-49e1-b15d-be954d162f88 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:47.280677 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0265adda-b616-4945-97e8-4f2a881f41bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:47.282164 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265adda-b616-4945-97e8-4f2a881f41bc None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:47.282661 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265adda-b616-4945-97e8-4f2a881f41bc None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:47.283819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265adda-b616-4945-97e8-4f2a881f41bc None admin] PATH_INFO: `/v3/projects/932a11c28fa2405793ce99d7097c63a5` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:47.285401 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0265adda-b616-4945-97e8-4f2a881f41bc None admin] RBAC: Authorizing `identity:get_project(project_id=932a11c28fa2405793ce99d7097c63a5)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:47.287219 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0265adda-b616-4945-97e8-4f2a881f41bc None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:47.289608 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 188/372] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:47 2022] GET /identity/v3/projects/932a11c28fa2405793ce99d7097c63a5 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:47.290604 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0265adda-b616-4945-97e8-4f2a881f41bc None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:47.307644 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ddb31-add6-4e64-a07e-aa1605e176df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:47.308232 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ddb31-add6-4e64-a07e-aa1605e176df None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:47.308514 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ddb31-add6-4e64-a07e-aa1605e176df None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:47.308791 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ddb31-add6-4e64-a07e-aa1605e176df None admin] PATH_INFO: `/v3/projects/5672fa915b944f15a9f6ebcb29dba655` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:47.309962 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ddb31-add6-4e64-a07e-aa1605e176df None admin] RBAC: Authorizing `identity:get_project(project_id=5672fa915b944f15a9f6ebcb29dba655)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:47.311448 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ddb31-add6-4e64-a07e-aa1605e176df None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:47.314833 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 185/373] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:47 2022] GET /identity/v3/projects/5672fa915b944f15a9f6ebcb29dba655 => generated 344 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:47.315720 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ddb31-add6-4e64-a07e-aa1605e176df None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:47.329302 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:47.417683 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0265adda-b616-4945-97e8-4f2a881f41bc None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:47.420187 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ddb31-add6-4e64-a07e-aa1605e176df None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:47.449065 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:47.462813 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8100e9d6-87ad-4780-ac11-0b368a59f931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:47.463744 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100e9d6-87ad-4780-ac11-0b368a59f931 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:47.464233 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100e9d6-87ad-4780-ac11-0b368a59f931 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:47.464953 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b9c8c-305c-40c6-b4d3-5dbae2702e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:47.465619 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b9c8c-305c-40c6-b4d3-5dbae2702e56 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:47.465898 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b9c8c-305c-40c6-b4d3-5dbae2702e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:47.466208 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b9c8c-305c-40c6-b4d3-5dbae2702e56 None admin] PATH_INFO: `/v3/groups/b4aab270fc5642f2b77d88507a8b6c2b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:47.467434 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b9c8c-305c-40c6-b4d3-5dbae2702e56 None admin] RBAC: Authorizing `identity:get_group(group_id=b4aab270fc5642f2b77d88507a8b6c2b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:47.467860 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100e9d6-87ad-4780-ac11-0b368a59f931 None admin] PATH_INFO: `/v3/projects/932a11c28fa2405793ce99d7097c63a5` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:47.469052 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b9c8c-305c-40c6-b4d3-5dbae2702e56 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:47.469549 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8100e9d6-87ad-4780-ac11-0b368a59f931 None admin] RBAC: Authorizing `identity:get_project(project_id=932a11c28fa2405793ce99d7097c63a5)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:47.471761 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8100e9d6-87ad-4780-ac11-0b368a59f931 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:47.472334 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 186/374] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Jul 27 04:08:47 2022] GET /identity/v3/groups/b4aab270fc5642f2b77d88507a8b6c2b => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:47.473121 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b9c8c-305c-40c6-b4d3-5dbae2702e56 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:47.474504 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 189/375] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:47 2022] GET /identity/v3/projects/932a11c28fa2405793ce99d7097c63a5 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:47.478335 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8100e9d6-87ad-4780-ac11-0b368a59f931 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:47.579332 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b9c8c-305c-40c6-b4d3-5dbae2702e56 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:47.581962 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:47.592312 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8100e9d6-87ad-4780-ac11-0b368a59f931 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:47.622076 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c734d2-bcd6-4380-86b0-f04c121fcb52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:47.622743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c734d2-bcd6-4380-86b0-f04c121fcb52 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:47.623022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c734d2-bcd6-4380-86b0-f04c121fcb52 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:47.623345 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c734d2-bcd6-4380-86b0-f04c121fcb52 None admin] PATH_INFO: `/v3/groups/d1809a3517b443ea9d3bd07e68ee4be9` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:47.630330 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c734d2-bcd6-4380-86b0-f04c121fcb52 None admin] RBAC: Authorizing `identity:get_group(group_id=d1809a3517b443ea9d3bd07e68ee4be9)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:47.631826 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c734d2-bcd6-4380-86b0-f04c121fcb52 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:47.633722 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 187/376] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Jul 27 04:08:47 2022] GET /identity/v3/groups/d1809a3517b443ea9d3bd07e68ee4be9 => generated 229 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:47.637997 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c734d2-bcd6-4380-86b0-f04c121fcb52 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:47.642068 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6053da2f-50e1-41f9-943a-57ee10115f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:47.643102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053da2f-50e1-41f9-943a-57ee10115f39 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:47.643755 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053da2f-50e1-41f9-943a-57ee10115f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:47.644253 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053da2f-50e1-41f9-943a-57ee10115f39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:47.645379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6053da2f-50e1-41f9-943a-57ee10115f39 None admin] RBAC: Adding query filter params (group.id=b4aab270fc5642f2b77d88507a8b6c2b, role.id=cdfc87279ea24b65bae3961035133725, scope.project.id=932a11c28fa2405793ce99d7097c63a5) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:47.645902 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6053da2f-50e1-41f9-943a-57ee10115f39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:47.647712 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6053da2f-50e1-41f9-943a-57ee10115f39 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:47.654670 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 190/377] 10.222.0.46 () {56 vars in 1376 bytes} [Wed Jul 27 04:08:47 2022] GET /identity/v3/role_assignments?group.id=b4aab270fc5642f2b77d88507a8b6c2b&scope.project.id=932a11c28fa2405793ce99d7097c63a5&role.id=cdfc87279ea24b65bae3961035133725 => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:08:47.656004 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6053da2f-50e1-41f9-943a-57ee10115f39 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:47.703506 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:47.755510 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c734d2-bcd6-4380-86b0-f04c121fcb52 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:47.793273 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6053da2f-50e1-41f9-943a-57ee10115f39 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:47.800599 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49634782-86f9-477c-9253-9b8eafdf9e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:47.800599 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49634782-86f9-477c-9253-9b8eafdf9e72 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:47.800599 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49634782-86f9-477c-9253-9b8eafdf9e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:47.800942 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49634782-86f9-477c-9253-9b8eafdf9e72 None admin] PATH_INFO: `/v3/groups/b4aab270fc5642f2b77d88507a8b6c2b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:47.802041 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49634782-86f9-477c-9253-9b8eafdf9e72 None admin] RBAC: Authorizing `identity:get_group(group_id=b4aab270fc5642f2b77d88507a8b6c2b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:47.803531 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49634782-86f9-477c-9253-9b8eafdf9e72 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:47.805447 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 188/378] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Jul 27 04:08:47 2022] GET /identity/v3/groups/b4aab270fc5642f2b77d88507a8b6c2b => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:47.807507 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e913c-be7f-49b3-abd9-cf0b37db43f6 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:47.807792 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e913c-be7f-49b3-abd9-cf0b37db43f6 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:47.808076 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e913c-be7f-49b3-abd9-cf0b37db43f6 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:47.809721 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 189/379] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:47.810912 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319ceb2-89bd-48e2-a960-9d21b779c1c5 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:47.811192 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319ceb2-89bd-48e2-a960-9d21b779c1c5 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:47.811578 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319ceb2-89bd-48e2-a960-9d21b779c1c5 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:47.813298 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 190/380] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:47.814127 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8319ceb2-89bd-48e2-a960-9d21b779c1c5 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:47.842541 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a7ad73-4bfd-48cd-b4ef-d4decb6dfca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:47.843174 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7ad73-4bfd-48cd-b4ef-d4decb6dfca8 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:47.843460 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7ad73-4bfd-48cd-b4ef-d4decb6dfca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:47.843736 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7ad73-4bfd-48cd-b4ef-d4decb6dfca8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:47.844544 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a7ad73-4bfd-48cd-b4ef-d4decb6dfca8 None admin] RBAC: Adding query filter params (group.id=d1809a3517b443ea9d3bd07e68ee4be9, role.id=cf771a8e94ee46149eb081347fea6d67, scope.project.id=5672fa915b944f15a9f6ebcb29dba655) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:47.844854 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a7ad73-4bfd-48cd-b4ef-d4decb6dfca8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:47.847449 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a7ad73-4bfd-48cd-b4ef-d4decb6dfca8 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:47.854145 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 191/381] 10.222.0.46 () {56 vars in 1376 bytes} [Wed Jul 27 04:08:47 2022] GET /identity/v3/role_assignments?group.id=d1809a3517b443ea9d3bd07e68ee4be9&scope.project.id=5672fa915b944f15a9f6ebcb29dba655&role.id=cf771a8e94ee46149eb081347fea6d67 => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:47.855114 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61341ae-59de-4435-ba68-ef8f81fa7f7d None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:47.855402 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61341ae-59de-4435-ba68-ef8f81fa7f7d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:47.855677 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61341ae-59de-4435-ba68-ef8f81fa7f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:47.876968 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-e61341ae-59de-4435-ba68-ef8f81fa7f7d None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:47.878515 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e61341ae-59de-4435-ba68-ef8f81fa7f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:47.899614 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:47.973561 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 192/382] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:47.974931 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8319ceb2-89bd-48e2-a960-9d21b779c1c5 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:47.976419 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe59f6-682f-463e-a525-9fad89a22383 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:47.976711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe59f6-682f-463e-a525-9fad89a22383 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:47.976985 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe59f6-682f-463e-a525-9fad89a22383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:47.997340 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-aefe59f6-682f-463e-a525-9fad89a22383 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:47.998583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-aefe59f6-682f-463e-a525-9fad89a22383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:48.013893 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:48.028945 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b73b63-caa4-4656-98da-0da463d052a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:48.030227 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b73b63-caa4-4656-98da-0da463d052a8 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:48.031088 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b73b63-caa4-4656-98da-0da463d052a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:48.032071 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b73b63-caa4-4656-98da-0da463d052a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:48.033965 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b73b63-caa4-4656-98da-0da463d052a8 None admin] RBAC: Adding query filter params (group.id=b4aab270fc5642f2b77d88507a8b6c2b, role.id=7b6f2ade82434ee79175385de17049eb, scope.project.id=932a11c28fa2405793ce99d7097c63a5) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:48.034541 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b73b63-caa4-4656-98da-0da463d052a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:48.036917 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b73b63-caa4-4656-98da-0da463d052a8 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:48.045052 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 191/383] 10.222.0.46 () {56 vars in 1376 bytes} [Wed Jul 27 04:08:47 2022] GET /identity/v3/role_assignments?group.id=b4aab270fc5642f2b77d88507a8b6c2b&scope.project.id=932a11c28fa2405793ce99d7097c63a5&role.id=7b6f2ade82434ee79175385de17049eb => generated 260 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:48.046431 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b73b63-caa4-4656-98da-0da463d052a8 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:48.086984 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 193/384] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:48.111770 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe59f6-682f-463e-a525-9fad89a22383 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:48.186649 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b73b63-caa4-4656-98da-0da463d052a8 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:48.215401 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:48.229773 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3aeaf-a06e-4904-abd2-d767d9b9e55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:48.230757 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3aeaf-a06e-4904-abd2-d767d9b9e55f None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:48.231670 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3aeaf-a06e-4904-abd2-d767d9b9e55f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:48.232658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3aeaf-a06e-4904-abd2-d767d9b9e55f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:48.238543 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe59f6-682f-463e-a525-9fad89a22383 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:48.246288 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e3aeaf-a06e-4904-abd2-d767d9b9e55f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:48.252674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-c9e3aeaf-a06e-4904-abd2-d767d9b9e55f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:48.252674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:48.252674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:48.252674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:48.252674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:48.252674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:48.252674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:48.252674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:48.252674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:48.252674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:48.252674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:48.252674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:48.252674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:48.252674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:48.252674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:48.252674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:48.253868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:48.253868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:48.253868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:48.253868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:48.253868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:48.253868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:48.253868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:48.253868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:48.253868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:48.253868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:48.253868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:48.253868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:48.253868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:48.253868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:48.253868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:48.253868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:48.254917 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:48.254917 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:48.254917 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:48.254917 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:48.254917 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:48.254917 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:48.254917 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:48.254917 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:48.254917 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:48.254917 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:48.254917 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:48.254917 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:08:48.257028 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 192/385] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:48.264153 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3aeaf-a06e-4904-abd2-d767d9b9e55f None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:48.286118 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61595fc8-5163-4b0a-ba19-fae39f9a18f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:48.286837 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61595fc8-5163-4b0a-ba19-fae39f9a18f3 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:48.287185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61595fc8-5163-4b0a-ba19-fae39f9a18f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:48.287435 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61595fc8-5163-4b0a-ba19-fae39f9a18f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:48.299369 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61595fc8-5163-4b0a-ba19-fae39f9a18f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:48.304616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-61595fc8-5163-4b0a-ba19-fae39f9a18f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 04:08:48.304616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:48.304616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:48.304616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:48.304616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:48.304616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:48.304616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:48.304616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:48.304616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:48.304616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:48.304616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:48.304616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:48.304616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:48.304616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:48.304616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:48.304616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:48.305750 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:48.305750 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:48.305750 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:48.305750 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:48.305750 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:48.305750 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:48.305750 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:48.305750 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:48.305750 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:48.305750 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:48.305750 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:48.305750 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:48.305750 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:48.305750 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:48.305750 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:48.305750 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:48.306669 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:48.306669 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:48.306669 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:48.306669 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:48.306669 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:48.306669 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:48.306669 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:48.306669 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:48.306669 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:48.306669 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:48.306669 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 04:08:48.306669 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:08:48.306669 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 194/386] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 27 04:08:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:08:48.312187 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61595fc8-5163-4b0a-ba19-fae39f9a18f3 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:48.360328 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:48.432387 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3aeaf-a06e-4904-abd2-d767d9b9e55f None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:48.437949 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61595fc8-5163-4b0a-ba19-fae39f9a18f3 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:48.484274 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed27ca-abe5-43f8-8cb7-38d3911ea1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:48.484728 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8393f42-57b6-40e4-867d-11a58b6a37a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:48.484945 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed27ca-abe5-43f8-8cb7-38d3911ea1ab None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:48.485179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed27ca-abe5-43f8-8cb7-38d3911ea1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:48.485355 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8393f42-57b6-40e4-867d-11a58b6a37a8 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:48.485503 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed27ca-abe5-43f8-8cb7-38d3911ea1ab None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:48.485653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8393f42-57b6-40e4-867d-11a58b6a37a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:48.489996 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8393f42-57b6-40e4-867d-11a58b6a37a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:48.490200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ed27ca-abe5-43f8-8cb7-38d3911ea1ab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:48.490200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ed27ca-abe5-43f8-8cb7-38d3911ea1ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:48.490510 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8393f42-57b6-40e4-867d-11a58b6a37a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:48.490510 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8393f42-57b6-40e4-867d-11a58b6a37a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:48.490807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ed27ca-abe5-43f8-8cb7-38d3911ea1ab None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:48.490949 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8393f42-57b6-40e4-867d-11a58b6a37a8 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:48.497626 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 195/387] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:08:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:48.505436 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 193/388] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 27 04:08:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:48.507205 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed27ca-abe5-43f8-8cb7-38d3911ea1ab None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:48.510413 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8393f42-57b6-40e4-867d-11a58b6a37a8 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:48.512885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:48.682000 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8393f42-57b6-40e4-867d-11a58b6a37a8 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:48.685951 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed27ca-abe5-43f8-8cb7-38d3911ea1ab None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:48.699246 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:48.718292 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72352b38-48bd-4325-a3e4-cf78e89a5b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:48.718947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72352b38-48bd-4325-a3e4-cf78e89a5b91 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:48.719229 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72352b38-48bd-4325-a3e4-cf78e89a5b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:48.719508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72352b38-48bd-4325-a3e4-cf78e89a5b91 None admin] PATH_INFO: `/v3/groups/b4aab270fc5642f2b77d88507a8b6c2b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:48.722815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72352b38-48bd-4325-a3e4-cf78e89a5b91 None admin] RBAC: Authorizing `identity:get_group(group_id=b4aab270fc5642f2b77d88507a8b6c2b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:48.724288 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72352b38-48bd-4325-a3e4-cf78e89a5b91 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:48.729997 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 194/389] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Jul 27 04:08:48 2022] GET /identity/v3/groups/b4aab270fc5642f2b77d88507a8b6c2b => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:48.731778 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72352b38-48bd-4325-a3e4-cf78e89a5b91 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:48.806728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08378eb6-5199-466f-a6fc-69035736aad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:48.807407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08378eb6-5199-466f-a6fc-69035736aad2 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:48.807688 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08378eb6-5199-466f-a6fc-69035736aad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:48.807962 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08378eb6-5199-466f-a6fc-69035736aad2 None admin] PATH_INFO: `/v3/groups/b4aab270fc5642f2b77d88507a8b6c2b` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:48.809214 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08378eb6-5199-466f-a6fc-69035736aad2 None admin] RBAC: Authorizing `identity:get_group(group_id=b4aab270fc5642f2b77d88507a8b6c2b)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:48.810695 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08378eb6-5199-466f-a6fc-69035736aad2 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:48.812616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 196/390] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Jul 27 04:08:48 2022] GET /identity/v3/groups/b4aab270fc5642f2b77d88507a8b6c2b => generated 226 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:48.818805 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08378eb6-5199-466f-a6fc-69035736aad2 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:48.842947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72352b38-48bd-4325-a3e4-cf78e89a5b91 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:48.880498 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fbb909-68de-45a5-8719-8a1ba6158932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:48.881152 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fbb909-68de-45a5-8719-8a1ba6158932 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:48.881430 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fbb909-68de-45a5-8719-8a1ba6158932 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:48.881702 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fbb909-68de-45a5-8719-8a1ba6158932 None admin] PATH_INFO: `/v3/projects/85168a036bf94350836ae48489878ed3` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:48.883274 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fbb909-68de-45a5-8719-8a1ba6158932 None admin] RBAC: Authorizing `identity:get_project(project_id=85168a036bf94350836ae48489878ed3)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:48.885049 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fbb909-68de-45a5-8719-8a1ba6158932 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:48.887789 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 195/391] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:48 2022] GET /identity/v3/projects/85168a036bf94350836ae48489878ed3 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:48.894063 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fbb909-68de-45a5-8719-8a1ba6158932 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:48.956663 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:49.005563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08378eb6-5199-466f-a6fc-69035736aad2 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:49.006294 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:49.057514 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf45c3e-d62d-45a8-ae1e-85d03664aa92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:49.058192 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf45c3e-d62d-45a8-ae1e-85d03664aa92 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:49.058472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf45c3e-d62d-45a8-ae1e-85d03664aa92 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:49.058745 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf45c3e-d62d-45a8-ae1e-85d03664aa92 None admin] PATH_INFO: `/v3/projects/85168a036bf94350836ae48489878ed3` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:49.061576 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf45c3e-d62d-45a8-ae1e-85d03664aa92 None admin] RBAC: Authorizing `identity:get_project(project_id=85168a036bf94350836ae48489878ed3)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:49.063315 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf45c3e-d62d-45a8-ae1e-85d03664aa92 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:49.069243 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 197/392] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:48 2022] GET /identity/v3/projects/85168a036bf94350836ae48489878ed3 => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:49.073726 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf45c3e-d62d-45a8-ae1e-85d03664aa92 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:49.102943 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fbb909-68de-45a5-8719-8a1ba6158932 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:49.212126 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404021e-5da6-4cba-af9e-46c10ec3bdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:49.212332 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf45c3e-d62d-45a8-ae1e-85d03664aa92 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:49.212742 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404021e-5da6-4cba-af9e-46c10ec3bdec None admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:49.213024 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404021e-5da6-4cba-af9e-46c10ec3bdec None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:49.213295 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404021e-5da6-4cba-af9e-46c10ec3bdec None admin] PATH_INFO: `/v3/projects/85168a036bf94350836ae48489878ed3/groups/b4aab270fc5642f2b77d88507a8b6c2b/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:49.218079 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7404021e-5da6-4cba-af9e-46c10ec3bdec None admin] RBAC: Authorizing `identity:create_grant(project_id=85168a036bf94350836ae48489878ed3, group_id=b4aab270fc5642f2b77d88507a8b6c2b, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:49.219530 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7404021e-5da6-4cba-af9e-46c10ec3bdec None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:49.254550 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 196/393] 10.222.0.46 () {58 vars in 1442 bytes} [Wed Jul 27 04:08:48 2022] PUT /identity/v3/projects/85168a036bf94350836ae48489878ed3/groups/b4aab270fc5642f2b77d88507a8b6c2b/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:08:49.260761 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0eaba-735a-46b7-bfe9-9ce1e8a8f98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:49.261480 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0eaba-735a-46b7-bfe9-9ce1e8a8f98b None admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:49.261767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0eaba-735a-46b7-bfe9-9ce1e8a8f98b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:49.262064 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0eaba-735a-46b7-bfe9-9ce1e8a8f98b None admin] PATH_INFO: `/v3/projects/85168a036bf94350836ae48489878ed3/groups/b4aab270fc5642f2b77d88507a8b6c2b/roles/cdfc87279ea24b65bae3961035133725` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:49.268776 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be0eaba-735a-46b7-bfe9-9ce1e8a8f98b None admin] RBAC: Authorizing `identity:create_grant(project_id=85168a036bf94350836ae48489878ed3, group_id=b4aab270fc5642f2b77d88507a8b6c2b, role_id=cdfc87279ea24b65bae3961035133725)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:49.270331 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be0eaba-735a-46b7-bfe9-9ce1e8a8f98b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:49.287066 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:49.296886 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 198/394] 10.222.0.46 () {58 vars in 1442 bytes} [Wed Jul 27 04:08:49 2022] PUT /identity/v3/projects/85168a036bf94350836ae48489878ed3/groups/b4aab270fc5642f2b77d88507a8b6c2b/roles/cdfc87279ea24b65bae3961035133725 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:08:50.037323 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ea03e-13aa-4c38-a492-01e3b17bda38 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:50.038340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ea03e-13aa-4c38-a492-01e3b17bda38 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:50.038825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ea03e-13aa-4c38-a492-01e3b17bda38 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:50.040678 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 197/395] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:50.045907 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1b89e-3c50-4abc-b8c8-a10d4e039e09 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:50.046563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1b89e-3c50-4abc-b8c8-a10d4e039e09 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:50.047048 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1b89e-3c50-4abc-b8c8-a10d4e039e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:50.078991 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-33a1b89e-3c50-4abc-b8c8-a10d4e039e09 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:50.080624 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-33a1b89e-3c50-4abc-b8c8-a10d4e039e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:50.166184 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 199/396] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:50.207667 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ea03e-13aa-4c38-a492-01e3b17bda38 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:50.332445 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ea03e-13aa-4c38-a492-01e3b17bda38 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:50.342320 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:50.372362 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea808a-3a21-40e1-be36-3c6d1f7d68b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:50.373374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea808a-3a21-40e1-be36-3c6d1f7d68b3 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:50.373903 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea808a-3a21-40e1-be36-3c6d1f7d68b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:50.374395 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea808a-3a21-40e1-be36-3c6d1f7d68b3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:50.389012 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ea808a-3a21-40e1-be36-3c6d1f7d68b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:50.394983 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-37ea808a-3a21-40e1-be36-3c6d1f7d68b3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 04:08:50.394983 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:50.394983 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:50.394983 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:50.394983 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:50.394983 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:50.394983 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:50.394983 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:50.394983 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:50.394983 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:50.394983 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:50.394983 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:50.394983 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:50.394983 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:50.394983 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:50.394983 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:50.395886 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:50.395886 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:50.395886 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:50.395886 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:50.395886 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:50.395886 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:50.395886 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:50.395886 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:50.395886 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:50.395886 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:50.395886 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:50.395886 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:50.395886 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:50.395886 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:50.395886 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:50.395886 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:50.396743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:50.396743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:50.396743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:50.396743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:50.396743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:50.396743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:50.396743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:50.396743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:50.396743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:50.396743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:50.396743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 04:08:50.396743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:08:50.396743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 198/397] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 27 04:08:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:50.402129 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1b89e-3c50-4abc-b8c8-a10d4e039e09 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:50.539493 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1b89e-3c50-4abc-b8c8-a10d4e039e09 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:50.569897 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:50.578794 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7b066-4f57-41de-b6bd-e35ffa68439b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:50.579425 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7b066-4f57-41de-b6bd-e35ffa68439b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:50.580273 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7b066-4f57-41de-b6bd-e35ffa68439b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:50.580571 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7b066-4f57-41de-b6bd-e35ffa68439b None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:50.581445 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7b066-4f57-41de-b6bd-e35ffa68439b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:50.581748 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7b066-4f57-41de-b6bd-e35ffa68439b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:50.583166 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7b066-4f57-41de-b6bd-e35ffa68439b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:50.597749 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 200/398] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:08:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:50.603239 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea808a-3a21-40e1-be36-3c6d1f7d68b3 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:50.707932 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14017aa-d09b-440f-a5ad-467aa04162e9 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:50.708252 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14017aa-d09b-440f-a5ad-467aa04162e9 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:50.708493 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14017aa-d09b-440f-a5ad-467aa04162e9 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:50.709850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 201/399] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:50.715353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38b0ad-7145-4243-a84e-95ce6ea42c31 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:50.715635 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38b0ad-7145-4243-a84e-95ce6ea42c31 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:50.715911 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38b0ad-7145-4243-a84e-95ce6ea42c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:50.743971 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-8b38b0ad-7145-4243-a84e-95ce6ea42c31 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:50.745181 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8b38b0ad-7145-4243-a84e-95ce6ea42c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:50.765923 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea808a-3a21-40e1-be36-3c6d1f7d68b3 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:50.825859 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4caae9-ae95-4611-9e98-75321de87447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:50.826551 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4caae9-ae95-4611-9e98-75321de87447 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:50.826837 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4caae9-ae95-4611-9e98-75321de87447 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:50.827116 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4caae9-ae95-4611-9e98-75321de87447 None admin] PATH_INFO: `/v3/groups/b4aab270fc5642f2b77d88507a8b6c2b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:50.828286 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4caae9-ae95-4611-9e98-75321de87447 None admin] RBAC: Authorizing `identity:get_group(group_id=b4aab270fc5642f2b77d88507a8b6c2b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:50.829893 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4caae9-ae95-4611-9e98-75321de87447 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:50.833849 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 199/400] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Jul 27 04:08:50 2022] GET /identity/v3/groups/b4aab270fc5642f2b77d88507a8b6c2b => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:50.839396 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4caae9-ae95-4611-9e98-75321de87447 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:50.884921 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:50.887082 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 202/401] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 04:08:50.926136 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b38b0ad-7145-4243-a84e-95ce6ea42c31 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:50.962503 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4caae9-ae95-4611-9e98-75321de87447 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:51.004255 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac861e-0fb0-4441-ac39-de1be2ec98a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:51.004935 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac861e-0fb0-4441-ac39-de1be2ec98a8 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:51.005371 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac861e-0fb0-4441-ac39-de1be2ec98a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:51.005665 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac861e-0fb0-4441-ac39-de1be2ec98a8 None admin] PATH_INFO: `/v3/projects/932a11c28fa2405793ce99d7097c63a5` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:51.007018 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ac861e-0fb0-4441-ac39-de1be2ec98a8 None admin] RBAC: Authorizing `identity:get_project(project_id=932a11c28fa2405793ce99d7097c63a5)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:51.008488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ac861e-0fb0-4441-ac39-de1be2ec98a8 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:51.010730 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 200/402] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:50 2022] GET /identity/v3/projects/932a11c28fa2405793ce99d7097c63a5 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:51.017023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac861e-0fb0-4441-ac39-de1be2ec98a8 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:51.124818 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b38b0ad-7145-4243-a84e-95ce6ea42c31 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:51.129657 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac861e-0fb0-4441-ac39-de1be2ec98a8 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:51.132799 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:51.162071 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:51.171384 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2692ec28-744c-401a-bd97-b1e37ffbe9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:51.172180 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692ec28-744c-401a-bd97-b1e37ffbe9b2 None admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:51.172414 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904f633-a2bc-4ad4-9acb-d8195e65687d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:51.172540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692ec28-744c-401a-bd97-b1e37ffbe9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:51.172737 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692ec28-744c-401a-bd97-b1e37ffbe9b2 None admin] PATH_INFO: `/v3/projects/932a11c28fa2405793ce99d7097c63a5/groups/b4aab270fc5642f2b77d88507a8b6c2b/roles/cdfc87279ea24b65bae3961035133725` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:51.173067 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904f633-a2bc-4ad4-9acb-d8195e65687d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:51.173348 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904f633-a2bc-4ad4-9acb-d8195e65687d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:51.173900 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904f633-a2bc-4ad4-9acb-d8195e65687d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:51.175247 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2692ec28-744c-401a-bd97-b1e37ffbe9b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=932a11c28fa2405793ce99d7097c63a5, group_id=b4aab270fc5642f2b77d88507a8b6c2b, role_id=cdfc87279ea24b65bae3961035133725)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:51.176703 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2692ec28-744c-401a-bd97-b1e37ffbe9b2 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:51.188725 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a904f633-a2bc-4ad4-9acb-d8195e65687d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:51.195848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-a904f633-a2bc-4ad4-9acb-d8195e65687d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:08:51.195848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:51.195848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:51.195848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:51.195848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:51.195848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:51.195848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:51.195848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:51.195848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:51.195848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:51.195848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:51.195848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:51.195848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:51.195848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:51.195848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:51.195848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:51.196784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:51.196784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:51.196784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:51.196784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:51.196784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:51.196784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:51.196784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:51.196784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:51.196784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:51.196784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:51.196784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:51.196784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:51.196784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:51.196784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:51.196784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:51.196784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:51.198286 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:51.198286 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:51.198286 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:51.198286 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:51.198286 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:51.198286 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:51.198286 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:51.198286 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:51.198286 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:51.198286 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:51.198286 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:08:51.198286 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:08:51.198286 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 203/403] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:08:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:08:51.199128 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 201/404] 10.222.0.46 () {58 vars in 1442 bytes} [Wed Jul 27 04:08:51 2022] PUT /identity/v3/projects/932a11c28fa2405793ce99d7097c63a5/groups/b4aab270fc5642f2b77d88507a8b6c2b/roles/cdfc87279ea24b65bae3961035133725 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:08:51.199204 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61776a05-c245-498d-8569-9ca289875c69 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:51.199204 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61776a05-c245-498d-8569-9ca289875c69 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:51.199204 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61776a05-c245-498d-8569-9ca289875c69 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:51.201121 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 204/405] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:51.211891 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61776a05-c245-498d-8569-9ca289875c69 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:51.213089 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469ff10-78be-4fee-bdb0-41a1f8d7d747 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:51.213377 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469ff10-78be-4fee-bdb0-41a1f8d7d747 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:51.213649 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469ff10-78be-4fee-bdb0-41a1f8d7d747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:51.258839 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-3469ff10-78be-4fee-bdb0-41a1f8d7d747 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:51.260123 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3469ff10-78be-4fee-bdb0-41a1f8d7d747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:51.374097 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61776a05-c245-498d-8569-9ca289875c69 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:51.416344 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a05ae1-36c2-4e55-bb88-61d63a84bae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:51.417141 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a05ae1-36c2-4e55-bb88-61d63a84bae7 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:51.417303 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 202/406] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:51.417479 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a05ae1-36c2-4e55-bb88-61d63a84bae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:51.417479 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a05ae1-36c2-4e55-bb88-61d63a84bae7 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:51.418225 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a05ae1-36c2-4e55-bb88-61d63a84bae7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:51.418582 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a05ae1-36c2-4e55-bb88-61d63a84bae7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:51.420047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a05ae1-36c2-4e55-bb88-61d63a84bae7 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:51.430655 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 205/407] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:51.441150 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3469ff10-78be-4fee-bdb0-41a1f8d7d747 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:51.447635 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a05ae1-36c2-4e55-bb88-61d63a84bae7 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:51.507031 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:51.562422 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:51.574953 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3469ff10-78be-4fee-bdb0-41a1f8d7d747 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:51.584118 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a05ae1-36c2-4e55-bb88-61d63a84bae7 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:51.614695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac1a6eb-8dd6-4f17-a728-22591f279d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:51.615723 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1a6eb-8dd6-4f17-a728-22591f279d33 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:51.616206 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1a6eb-8dd6-4f17-a728-22591f279d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:51.616702 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1a6eb-8dd6-4f17-a728-22591f279d33 None admin] PATH_INFO: `/v3/groups/d1809a3517b443ea9d3bd07e68ee4be9` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:51.618189 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac1a6eb-8dd6-4f17-a728-22591f279d33 None admin] RBAC: Authorizing `identity:get_group(group_id=d1809a3517b443ea9d3bd07e68ee4be9)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:51.619838 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac1a6eb-8dd6-4f17-a728-22591f279d33 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:51.622034 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 203/408] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Jul 27 04:08:51 2022] GET /identity/v3/groups/d1809a3517b443ea9d3bd07e68ee4be9 => generated 229 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:51.630010 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29e3cd-7fe3-46e2-aca6-92f8b1d21e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:51.630010 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29e3cd-7fe3-46e2-aca6-92f8b1d21e2c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:51.630010 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29e3cd-7fe3-46e2-aca6-92f8b1d21e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:51.630010 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29e3cd-7fe3-46e2-aca6-92f8b1d21e2c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:51.631245 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac1a6eb-8dd6-4f17-a728-22591f279d33 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:51.652972 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b29e3cd-7fe3-46e2-aca6-92f8b1d21e2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:51.666828 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-7b29e3cd-7fe3-46e2-aca6-92f8b1d21e2c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:51.666828 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:51.666828 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:51.666828 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:51.666828 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:51.666828 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:51.666828 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:51.666828 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:51.666828 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:51.666828 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:51.666828 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:51.666828 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:51.666828 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:51.666828 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:51.666828 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:51.666828 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:51.667721 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:51.667721 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:51.667721 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:51.667721 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:51.667721 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:51.667721 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:51.667721 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:51.667721 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:51.667721 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:51.667721 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:51.667721 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:51.667721 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:51.667721 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:51.667721 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:51.667721 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:51.667721 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:51.668507 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:51.668507 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:51.668507 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:51.668507 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:51.668507 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:51.668507 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:51.668507 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:51.668507 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:51.668507 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:51.668507 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:51.668507 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:51.668507 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:08:51.671531 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 206/409] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:08:51.695035 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29e3cd-7fe3-46e2-aca6-92f8b1d21e2c None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:51.814820 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac1a6eb-8dd6-4f17-a728-22591f279d33 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:51.851579 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33203f0b-b541-4e36-b880-5018b36fd3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:51.852243 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33203f0b-b541-4e36-b880-5018b36fd3d3 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:51.852519 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33203f0b-b541-4e36-b880-5018b36fd3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:51.852789 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33203f0b-b541-4e36-b880-5018b36fd3d3 None admin] PATH_INFO: `/v3/projects/5672fa915b944f15a9f6ebcb29dba655` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:51.853958 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33203f0b-b541-4e36-b880-5018b36fd3d3 None admin] RBAC: Authorizing `identity:get_project(project_id=5672fa915b944f15a9f6ebcb29dba655)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:51.855545 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33203f0b-b541-4e36-b880-5018b36fd3d3 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:51.857505 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 204/410] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:51 2022] GET /identity/v3/projects/5672fa915b944f15a9f6ebcb29dba655 => generated 344 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:51.863331 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33203f0b-b541-4e36-b880-5018b36fd3d3 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:51.875885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29e3cd-7fe3-46e2-aca6-92f8b1d21e2c None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:51.879803 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:51.917478 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:51.953917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d7e82-8f25-47cf-8c70-344084f3225b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:51.955298 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d7e82-8f25-47cf-8c70-344084f3225b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:51.955795 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d7e82-8f25-47cf-8c70-344084f3225b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:51.956278 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d7e82-8f25-47cf-8c70-344084f3225b None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:51.957233 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d7e82-8f25-47cf-8c70-344084f3225b None admin] RBAC: Adding query filter params (name=member) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:51.957747 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d7e82-8f25-47cf-8c70-344084f3225b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:51.959368 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d7e82-8f25-47cf-8c70-344084f3225b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:51.969091 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 207/411] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 27 04:08:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:08:51.971251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499c35d-139f-448a-9088-1f9531172334 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:51.971744 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499c35d-139f-448a-9088-1f9531172334 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:51.972247 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499c35d-139f-448a-9088-1f9531172334 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:51.977872 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 208/412] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:51.979036 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499c35d-139f-448a-9088-1f9531172334 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:51.998202 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33203f0b-b541-4e36-b880-5018b36fd3d3 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:52.035803 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f018c9-edb6-4eda-9d86-33308bb27d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:52.036484 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f018c9-edb6-4eda-9d86-33308bb27d75 None admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:52.036762 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f018c9-edb6-4eda-9d86-33308bb27d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:52.037030 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f018c9-edb6-4eda-9d86-33308bb27d75 None admin] PATH_INFO: `/v3/projects/5672fa915b944f15a9f6ebcb29dba655/groups/d1809a3517b443ea9d3bd07e68ee4be9/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:52.039016 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f018c9-edb6-4eda-9d86-33308bb27d75 None admin] RBAC: Authorizing `identity:create_grant(project_id=5672fa915b944f15a9f6ebcb29dba655, group_id=d1809a3517b443ea9d3bd07e68ee4be9, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:52.040463 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f018c9-edb6-4eda-9d86-33308bb27d75 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:52.055040 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 205/413] 10.222.0.46 () {58 vars in 1442 bytes} [Wed Jul 27 04:08:51 2022] PUT /identity/v3/projects/5672fa915b944f15a9f6ebcb29dba655/groups/d1809a3517b443ea9d3bd07e68ee4be9/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:08:52.056357 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a181741-e0d3-4006-856f-baf2917bc78f None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:52.056652 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a181741-e0d3-4006-856f-baf2917bc78f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:52.056931 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a181741-e0d3-4006-856f-baf2917bc78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:52.078753 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-2a181741-e0d3-4006-856f-baf2917bc78f None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:52.080887 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2a181741-e0d3-4006-856f-baf2917bc78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:52.110039 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499c35d-139f-448a-9088-1f9531172334 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:52.151050 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcab59c-6d65-4fb7-9b60-9de68bbb4c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:52.152137 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcab59c-6d65-4fb7-9b60-9de68bbb4c68 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:52.152711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcab59c-6d65-4fb7-9b60-9de68bbb4c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:52.153198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcab59c-6d65-4fb7-9b60-9de68bbb4c68 None admin] PATH_INFO: `/v3/groups/b4aab270fc5642f2b77d88507a8b6c2b` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:52.155050 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcab59c-6d65-4fb7-9b60-9de68bbb4c68 None admin] RBAC: Authorizing `identity:get_group(group_id=b4aab270fc5642f2b77d88507a8b6c2b)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:52.156917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcab59c-6d65-4fb7-9b60-9de68bbb4c68 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:52.160178 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 209/414] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Jul 27 04:08:51 2022] GET /identity/v3/groups/b4aab270fc5642f2b77d88507a8b6c2b => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:52.161915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb48c3a-464b-4f09-aa3c-c778a0d85123 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:52.162425 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb48c3a-464b-4f09-aa3c-c778a0d85123 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:52.162938 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb48c3a-464b-4f09-aa3c-c778a0d85123 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:52.164816 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 210/415] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:52.165853 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb48c3a-464b-4f09-aa3c-c778a0d85123 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:52.186592 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 206/416] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:52.186592 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302819a-5aa1-4779-bf23-5ca810a6ce4f None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:52.186592 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302819a-5aa1-4779-bf23-5ca810a6ce4f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:52.186592 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302819a-5aa1-4779-bf23-5ca810a6ce4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:52.208249 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-6302819a-5aa1-4779-bf23-5ca810a6ce4f None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:52.209746 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-6302819a-5aa1-4779-bf23-5ca810a6ce4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:52.226272 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:52.230227 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:52.311452 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 207/417] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:52.312795 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6302819a-5aa1-4779-bf23-5ca810a6ce4f None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:52.333908 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb48c3a-464b-4f09-aa3c-c778a0d85123 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:52.376784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd8ddc-71be-4b1a-be08-1138365014cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:52.377489 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd8ddc-71be-4b1a-be08-1138365014cf None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:52.377781 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd8ddc-71be-4b1a-be08-1138365014cf None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:52.378089 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd8ddc-71be-4b1a-be08-1138365014cf None admin] PATH_INFO: `/v3/projects/932a11c28fa2405793ce99d7097c63a5` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:52.379259 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd8ddc-71be-4b1a-be08-1138365014cf None admin] RBAC: Authorizing `identity:get_project(project_id=932a11c28fa2405793ce99d7097c63a5)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:52.381301 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd8ddc-71be-4b1a-be08-1138365014cf None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:52.383981 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 211/418] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:52 2022] GET /identity/v3/projects/932a11c28fa2405793ce99d7097c63a5 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:52.389558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd8ddc-71be-4b1a-be08-1138365014cf None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:52.480916 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6302819a-5aa1-4779-bf23-5ca810a6ce4f None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:52.512378 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:52.515748 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd8ddc-71be-4b1a-be08-1138365014cf None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:52.531852 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd1a23-989d-46e2-9759-a3537b8b1faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:52.533480 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd1a23-989d-46e2-9759-a3537b8b1faa None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:52.533765 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd1a23-989d-46e2-9759-a3537b8b1faa None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:52.534452 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd1a23-989d-46e2-9759-a3537b8b1faa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:52.549396 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd1a23-989d-46e2-9759-a3537b8b1faa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:52.553866 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f73ab7-3e95-48f8-a7bd-c3d9f8bcc0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:52.554552 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f73ab7-3e95-48f8-a7bd-c3d9f8bcc0c1 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:52.554838 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f73ab7-3e95-48f8-a7bd-c3d9f8bcc0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:52.554940 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-4ebd1a23-989d-46e2-9759-a3537b8b1faa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:52.554940 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:52.554940 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:52.554940 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:52.554940 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:52.554940 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:52.554940 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:52.554940 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:52.554940 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:52.554940 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:52.554940 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:52.554940 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:52.554940 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:52.554940 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:52.554940 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:52.554940 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:52.555760 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:52.555760 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:52.555760 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:52.555760 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:52.555760 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:52.555760 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:52.555760 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:52.555760 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:52.555760 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:52.555760 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:52.555760 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:52.555760 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:52.555760 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:52.555760 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:52.555760 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:52.555760 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:52.556540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:52.556540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:52.556540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:52.556540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:52.556540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:52.556540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:52.556540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:52.556540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:52.556540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:52.556540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:52.556540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:52.556540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:08:52.557131 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f73ab7-3e95-48f8-a7bd-c3d9f8bcc0c1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:52.560882 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 208/419] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:52.560882 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd1a23-989d-46e2-9759-a3537b8b1faa None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:52.570988 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f73ab7-3e95-48f8-a7bd-c3d9f8bcc0c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:52.575845 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:52.577799 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-d3f73ab7-3e95-48f8-a7bd-c3d9f8bcc0c1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 04:08:52.577799 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:52.577799 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:52.577799 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:52.577799 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:52.577799 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:52.577799 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:52.577799 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:52.577799 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:52.577799 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:52.577799 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:52.577799 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:52.577799 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:52.577799 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:52.577799 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:52.577799 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:52.580819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:52.580819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:52.580819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:52.580819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:52.580819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:52.580819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:52.580819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:52.580819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:52.580819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:52.580819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:52.580819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:52.580819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:52.580819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:52.580819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:52.580819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:52.580819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:52.582634 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:52.582634 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:52.582634 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:52.582634 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:52.582634 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:52.582634 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:52.582634 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:52.582634 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:52.582634 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:52.582634 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:52.582634 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 04:08:52.582634 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:08:52.582634 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 212/420] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 27 04:08:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:52.582634 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f73ab7-3e95-48f8-a7bd-c3d9f8bcc0c1 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:52.692174 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f73ab7-3e95-48f8-a7bd-c3d9f8bcc0c1 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:52.697946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd1a23-989d-46e2-9759-a3537b8b1faa None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:52.727572 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd1ceb5-bddc-470c-8212-68dbee73ed6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:52.728230 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1ceb5-bddc-470c-8212-68dbee73ed6c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:52.728649 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1ceb5-bddc-470c-8212-68dbee73ed6c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:52.728928 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1ceb5-bddc-470c-8212-68dbee73ed6c None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:52.729694 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd1ceb5-bddc-470c-8212-68dbee73ed6c None admin] RBAC: Adding query filter params (name=member) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:52.730032 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd1ceb5-bddc-470c-8212-68dbee73ed6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:52.731611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd1ceb5-bddc-470c-8212-68dbee73ed6c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:52.742729 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 213/421] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 27 04:08:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:52.743719 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd1ceb5-bddc-470c-8212-68dbee73ed6c None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:52.749147 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b60a3-da0a-46b2-989a-067a3d13edc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:52.749865 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b60a3-da0a-46b2-989a-067a3d13edc3 None admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:52.750190 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b60a3-da0a-46b2-989a-067a3d13edc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:52.750475 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b60a3-da0a-46b2-989a-067a3d13edc3 None admin] PATH_INFO: `/v3/projects/932a11c28fa2405793ce99d7097c63a5/groups/b4aab270fc5642f2b77d88507a8b6c2b/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:52.752601 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b60a3-da0a-46b2-989a-067a3d13edc3 None admin] RBAC: Authorizing `identity:create_grant(project_id=932a11c28fa2405793ce99d7097c63a5, group_id=b4aab270fc5642f2b77d88507a8b6c2b, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:52.754626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b60a3-da0a-46b2-989a-067a3d13edc3 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:52.773906 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:52.775843 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 209/422] 10.222.0.46 () {58 vars in 1442 bytes} [Wed Jul 27 04:08:52 2022] PUT /identity/v3/projects/932a11c28fa2405793ce99d7097c63a5/groups/b4aab270fc5642f2b77d88507a8b6c2b/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:08:52.776682 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b60a3-da0a-46b2-989a-067a3d13edc3 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:52.870952 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:52.944857 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd1ceb5-bddc-470c-8212-68dbee73ed6c None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:52.956361 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b60a3-da0a-46b2-989a-067a3d13edc3 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:52.981967 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1613b44-21c1-443b-b122-7dce214b416c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:52.982636 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1613b44-21c1-443b-b122-7dce214b416c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:52.982930 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1613b44-21c1-443b-b122-7dce214b416c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:52.983213 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1613b44-21c1-443b-b122-7dce214b416c None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:52.983948 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1613b44-21c1-443b-b122-7dce214b416c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:52.984258 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1613b44-21c1-443b-b122-7dce214b416c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:52.985687 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1613b44-21c1-443b-b122-7dce214b416c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:52.995992 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 214/423] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:08:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:52.999631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de348b6d-3710-4414-834e-cff6262a6c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:53.000641 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de348b6d-3710-4414-834e-cff6262a6c15 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:53.001094 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de348b6d-3710-4414-834e-cff6262a6c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:53.001538 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1613b44-21c1-443b-b122-7dce214b416c None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:53.001793 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de348b6d-3710-4414-834e-cff6262a6c15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:53.088841 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de348b6d-3710-4414-834e-cff6262a6c15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:53.090531 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de348b6d-3710-4414-834e-cff6262a6c15 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:53.129682 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:53.168127 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1613b44-21c1-443b-b122-7dce214b416c None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:53.172837 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:53.210778 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 210/424] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:08:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:08:53.217453 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de348b6d-3710-4414-834e-cff6262a6c15 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:53.263763 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498aebb-9146-4cec-9f06-1465fa436cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:53.264310 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498aebb-9146-4cec-9f06-1465fa436cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:53.264589 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498aebb-9146-4cec-9f06-1465fa436cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:53.264860 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498aebb-9146-4cec-9f06-1465fa436cf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:53.353736 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de348b6d-3710-4414-834e-cff6262a6c15 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:53.404336 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d498aebb-9146-4cec-9f06-1465fa436cf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:53.405945 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d498aebb-9146-4cec-9f06-1465fa436cf1 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:53.407225 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c39adb9-7b0b-4b7a-8b18-b8d285dff6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:53.408424 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39adb9-7b0b-4b7a-8b18-b8d285dff6a9 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:53.408704 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39adb9-7b0b-4b7a-8b18-b8d285dff6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:53.408975 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39adb9-7b0b-4b7a-8b18-b8d285dff6a9 None admin] PATH_INFO: `/v3/projects/85168a036bf94350836ae48489878ed3` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:53.410191 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c39adb9-7b0b-4b7a-8b18-b8d285dff6a9 None admin] RBAC: Authorizing `identity:get_project(project_id=85168a036bf94350836ae48489878ed3)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:53.411609 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c39adb9-7b0b-4b7a-8b18-b8d285dff6a9 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:53.413631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 211/425] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:53 2022] GET /identity/v3/projects/85168a036bf94350836ae48489878ed3 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:53.418825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c39adb9-7b0b-4b7a-8b18-b8d285dff6a9 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:53.500989 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:53.518433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:53.537017 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 215/426] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:08:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 04:08:53.546948 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498aebb-9146-4cec-9f06-1465fa436cf1 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:53.598706 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c39adb9-7b0b-4b7a-8b18-b8d285dff6a9 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:53.643538 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f578f-5114-481b-9dda-d95c3b39a669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:53.644729 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f578f-5114-481b-9dda-d95c3b39a669 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:53.645534 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f578f-5114-481b-9dda-d95c3b39a669 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:53.646306 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f578f-5114-481b-9dda-d95c3b39a669 None admin] PATH_INFO: `/v3/groups/b4aab270fc5642f2b77d88507a8b6c2b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:53.647719 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f578f-5114-481b-9dda-d95c3b39a669 None admin] RBAC: Authorizing `identity:get_group(group_id=b4aab270fc5642f2b77d88507a8b6c2b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:53.650172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f578f-5114-481b-9dda-d95c3b39a669 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:53.652460 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 212/427] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Jul 27 04:08:53 2022] GET /identity/v3/groups/b4aab270fc5642f2b77d88507a8b6c2b => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:53.658865 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3806498-7db1-4f85-bde2-29c933a8515d None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:53.658865 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3806498-7db1-4f85-bde2-29c933a8515d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:53.658865 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3806498-7db1-4f85-bde2-29c933a8515d None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:53.662043 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 213/428] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:53.663135 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3806498-7db1-4f85-bde2-29c933a8515d None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:53.698930 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498aebb-9146-4cec-9f06-1465fa436cf1 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:53.749777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ddbbb-303f-4b77-832e-6ebac5a44c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:53.750468 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ddbbb-303f-4b77-832e-6ebac5a44c40 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:53.750762 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ddbbb-303f-4b77-832e-6ebac5a44c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:53.751140 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ddbbb-303f-4b77-832e-6ebac5a44c40 None admin] PATH_INFO: `/v3/projects/85168a036bf94350836ae48489878ed3` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:53.753069 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16ddbbb-303f-4b77-832e-6ebac5a44c40 None admin] RBAC: Authorizing `identity:get_project(project_id=85168a036bf94350836ae48489878ed3)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:53.754809 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16ddbbb-303f-4b77-832e-6ebac5a44c40 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:53.762063 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 216/429] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:53 2022] GET /identity/v3/projects/85168a036bf94350836ae48489878ed3 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:53.762063 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d0ed2-9e02-4bf3-acae-6157da89e5d4 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:53.762063 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d0ed2-9e02-4bf3-acae-6157da89e5d4 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:53.762063 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d0ed2-9e02-4bf3-acae-6157da89e5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:53.783009 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-3c1d0ed2-9e02-4bf3-acae-6157da89e5d4 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:53.784291 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-3c1d0ed2-9e02-4bf3-acae-6157da89e5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:53.804163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3806498-7db1-4f85-bde2-29c933a8515d None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:53.812240 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:53.837000 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:53.847734 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d2c8e-6273-4ef8-af77-55128c05ae82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:53.848587 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d2c8e-6273-4ef8-af77-55128c05ae82 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:53.849017 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d2c8e-6273-4ef8-af77-55128c05ae82 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:53.849483 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d2c8e-6273-4ef8-af77-55128c05ae82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:53.850612 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d2c8e-6273-4ef8-af77-55128c05ae82 None admin] RBAC: Adding query filter params (group.id=b4aab270fc5642f2b77d88507a8b6c2b, role.id=7b6f2ade82434ee79175385de17049eb, scope.project.id=85168a036bf94350836ae48489878ed3) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:53.851213 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d2c8e-6273-4ef8-af77-55128c05ae82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:53.852956 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d2c8e-6273-4ef8-af77-55128c05ae82 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:53.862005 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 217/430] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:53.862005 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d0ed2-9e02-4bf3-acae-6157da89e5d4 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:53.865060 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 214/431] 10.222.0.46 () {56 vars in 1376 bytes} [Wed Jul 27 04:08:53 2022] GET /identity/v3/role_assignments?group.id=b4aab270fc5642f2b77d88507a8b6c2b&scope.project.id=85168a036bf94350836ae48489878ed3&role.id=7b6f2ade82434ee79175385de17049eb => generated 611 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:53.930003 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d2c8e-6273-4ef8-af77-55128c05ae82 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:54.057476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d0ed2-9e02-4bf3-acae-6157da89e5d4 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:54.093082 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f5ea1-4395-4f83-8c11-959f9fbe36fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:54.093530 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f5ea1-4395-4f83-8c11-959f9fbe36fa None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:54.093822 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f5ea1-4395-4f83-8c11-959f9fbe36fa None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:54.094135 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f5ea1-4395-4f83-8c11-959f9fbe36fa None admin] PATH_INFO: `/v3/groups/b4aab270fc5642f2b77d88507a8b6c2b` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:54.096007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f5ea1-4395-4f83-8c11-959f9fbe36fa None admin] RBAC: Authorizing `identity:get_group(group_id=b4aab270fc5642f2b77d88507a8b6c2b)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:54.097549 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f5ea1-4395-4f83-8c11-959f9fbe36fa None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:54.100205 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 218/432] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Jul 27 04:08:53 2022] GET /identity/v3/groups/b4aab270fc5642f2b77d88507a8b6c2b => generated 226 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:54.107602 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f5ea1-4395-4f83-8c11-959f9fbe36fa None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:54.110629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d2c8e-6273-4ef8-af77-55128c05ae82 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:54.155570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30beec98-0fbb-44f4-98f4-ae179409e403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:54.156677 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30beec98-0fbb-44f4-98f4-ae179409e403 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:54.157193 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30beec98-0fbb-44f4-98f4-ae179409e403 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:54.157721 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30beec98-0fbb-44f4-98f4-ae179409e403 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:54.164184 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:54.172123 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30beec98-0fbb-44f4-98f4-ae179409e403 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:54.179666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-30beec98-0fbb-44f4-98f4-ae179409e403 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 04:08:54.179666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:54.179666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:54.179666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:54.179666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:54.179666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:54.179666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:54.179666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:54.179666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:54.179666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:54.179666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:54.179666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:54.179666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:54.179666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:54.179666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:54.179666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:54.180584 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:54.180584 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:54.180584 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:54.180584 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:54.180584 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:54.180584 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:54.180584 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:54.180584 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:54.180584 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:54.180584 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:54.180584 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:54.180584 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:54.180584 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:54.180584 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:54.180584 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:54.180584 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:54.181385 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:54.181385 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:54.181385 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:54.181385 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:54.181385 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:54.181385 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:54.181385 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:54.181385 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:54.181385 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:54.181385 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:54.181385 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 04:08:54.181385 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:08:54.184854 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 215/433] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 27 04:08:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:08:54.193715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30beec98-0fbb-44f4-98f4-ae179409e403 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:54.230178 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f5ea1-4395-4f83-8c11-959f9fbe36fa None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:54.230601 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:54.271195 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee1b866-a097-4a59-85de-547674b992b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:54.271826 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee1b866-a097-4a59-85de-547674b992b2 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:54.272112 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee1b866-a097-4a59-85de-547674b992b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:54.272453 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee1b866-a097-4a59-85de-547674b992b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:54.273261 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee1b866-a097-4a59-85de-547674b992b2 None admin] RBAC: Adding query filter params (group.id=b4aab270fc5642f2b77d88507a8b6c2b, role.id=cdfc87279ea24b65bae3961035133725, scope.project.id=85168a036bf94350836ae48489878ed3) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:54.273620 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee1b866-a097-4a59-85de-547674b992b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:54.275185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee1b866-a097-4a59-85de-547674b992b2 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:54.284127 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 219/434] 10.222.0.46 () {56 vars in 1376 bytes} [Wed Jul 27 04:08:54 2022] GET /identity/v3/role_assignments?group.id=b4aab270fc5642f2b77d88507a8b6c2b&scope.project.id=85168a036bf94350836ae48489878ed3&role.id=cdfc87279ea24b65bae3961035133725 => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:54.318265 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30beec98-0fbb-44f4-98f4-ae179409e403 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:54.348007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a215ff9b-a5a2-476d-bde5-949c50b08b49 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:54.348332 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a215ff9b-a5a2-476d-bde5-949c50b08b49 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:54.348567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a215ff9b-a5a2-476d-bde5-949c50b08b49 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:54.349996 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 220/435] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 04:08:54.358023 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db586cf-35d1-47fe-abd7-54e2b79e2e1d None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:54.358023 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db586cf-35d1-47fe-abd7-54e2b79e2e1d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:54.358023 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db586cf-35d1-47fe-abd7-54e2b79e2e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:54.367132 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8512765b-527c-4cb8-acdb-128137224cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:54.368152 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512765b-527c-4cb8-acdb-128137224cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:54.368643 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512765b-527c-4cb8-acdb-128137224cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:54.369129 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512765b-527c-4cb8-acdb-128137224cc2 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:54.370111 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8512765b-527c-4cb8-acdb-128137224cc2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:54.370681 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8512765b-527c-4cb8-acdb-128137224cc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:54.372347 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8512765b-527c-4cb8-acdb-128137224cc2 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:54.377446 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-9db586cf-35d1-47fe-abd7-54e2b79e2e1d None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:54.378558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-9db586cf-35d1-47fe-abd7-54e2b79e2e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:54.386440 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 216/436] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:08:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:54.394403 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8512765b-527c-4cb8-acdb-128137224cc2 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:54.478814 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 221/437] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:54.504390 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db586cf-35d1-47fe-abd7-54e2b79e2e1d None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:54.525438 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:54.535538 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8512765b-527c-4cb8-acdb-128137224cc2 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:54.571728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:54.582942 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e4aef-4ec2-4fab-bb8f-168d9b09413d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:54.582942 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e4aef-4ec2-4fab-bb8f-168d9b09413d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:54.583195 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e4aef-4ec2-4fab-bb8f-168d9b09413d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:54.583426 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e4aef-4ec2-4fab-bb8f-168d9b09413d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:54.647090 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db586cf-35d1-47fe-abd7-54e2b79e2e1d None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:54.676985 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41e4aef-4ec2-4fab-bb8f-168d9b09413d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:54.678744 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41e4aef-4ec2-4fab-bb8f-168d9b09413d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:54.687578 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea521e-c538-4657-b43f-f13060c4f930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:54.688493 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea521e-c538-4657-b43f-f13060c4f930 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:54.688770 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea521e-c538-4657-b43f-f13060c4f930 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:54.689568 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea521e-c538-4657-b43f-f13060c4f930 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:54.705492 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ea521e-c538-4657-b43f-f13060c4f930 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:54.710451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-e2ea521e-c538-4657-b43f-f13060c4f930 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:08:54.710451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:54.710451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:54.710451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:54.710451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:54.710451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:54.710451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:54.710451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:54.710451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:54.710451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:54.710451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:54.710451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:54.710451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:54.710451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:54.710451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:54.710451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:54.711343 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:54.711343 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:54.711343 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:54.711343 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:54.711343 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:54.711343 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:54.711343 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:54.711343 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:54.711343 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:54.711343 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:54.711343 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:54.711343 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:54.711343 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:54.711343 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:54.711343 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:54.711343 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:54.712133 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:54.712133 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:54.712133 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:54.712133 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:54.712133 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:54.712133 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:54.712133 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:54.712133 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:54.712133 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:54.712133 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:54.712133 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:08:54.712133 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:08:54.716601 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 222/438] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:08:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:54.718015 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea521e-c538-4657-b43f-f13060c4f930 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:54.804330 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:54.817921 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 217/439] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:08:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 04:08:54.824689 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e4aef-4ec2-4fab-bb8f-168d9b09413d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:54.848745 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea521e-c538-4657-b43f-f13060c4f930 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:54.854904 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:54.903683 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7a466-6fe1-4ac5-a6ff-e482431bdde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:54.904491 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7a466-6fe1-4ac5-a6ff-e482431bdde4 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:54.904870 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7a466-6fe1-4ac5-a6ff-e482431bdde4 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:54.905236 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7a466-6fe1-4ac5-a6ff-e482431bdde4 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:54.905990 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e7a466-6fe1-4ac5-a6ff-e482431bdde4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:54.906303 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e7a466-6fe1-4ac5-a6ff-e482431bdde4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:54.908237 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e7a466-6fe1-4ac5-a6ff-e482431bdde4 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:54.919307 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 223/440] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:08:54.925350 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7a466-6fe1-4ac5-a6ff-e482431bdde4 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:54.962523 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e4aef-4ec2-4fab-bb8f-168d9b09413d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:54.999674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf20c58-200a-4976-948d-6009a3211d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:55.000327 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf20c58-200a-4976-948d-6009a3211d04 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:55.000612 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf20c58-200a-4976-948d-6009a3211d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:55.000892 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf20c58-200a-4976-948d-6009a3211d04 None admin] PATH_INFO: `/v3/projects/932a11c28fa2405793ce99d7097c63a5` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:55.002685 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf20c58-200a-4976-948d-6009a3211d04 None admin] RBAC: Authorizing `identity:get_project(project_id=932a11c28fa2405793ce99d7097c63a5)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:55.004122 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf20c58-200a-4976-948d-6009a3211d04 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:55.005954 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 218/441] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:54 2022] GET /identity/v3/projects/932a11c28fa2405793ce99d7097c63a5 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:08:55.011126 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf20c58-200a-4976-948d-6009a3211d04 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:55.034114 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7a466-6fe1-4ac5-a6ff-e482431bdde4 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:55.070738 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:55.071742 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b962fb-0c07-400f-8379-d58952870789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:55.072369 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b962fb-0c07-400f-8379-d58952870789 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:55.072654 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b962fb-0c07-400f-8379-d58952870789 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:55.072933 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b962fb-0c07-400f-8379-d58952870789 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:55.107343 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:55.112434 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf20c58-200a-4976-948d-6009a3211d04 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:55.140614 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b962fb-0c07-400f-8379-d58952870789 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:55.142686 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b962fb-0c07-400f-8379-d58952870789 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:55.146877 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb55534e-85f1-4729-bbdf-5c0bce4bb67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:55.147518 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55534e-85f1-4729-bbdf-5c0bce4bb67c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:55.147796 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55534e-85f1-4729-bbdf-5c0bce4bb67c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:55.148067 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55534e-85f1-4729-bbdf-5c0bce4bb67c None admin] PATH_INFO: `/v3/groups/b4aab270fc5642f2b77d88507a8b6c2b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:55.149209 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb55534e-85f1-4729-bbdf-5c0bce4bb67c None admin] RBAC: Authorizing `identity:get_group(group_id=b4aab270fc5642f2b77d88507a8b6c2b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:55.150669 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb55534e-85f1-4729-bbdf-5c0bce4bb67c None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:55.152258 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 219/442] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Jul 27 04:08:55 2022] GET /identity/v3/groups/b4aab270fc5642f2b77d88507a8b6c2b => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:08:55.153360 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737160a-340e-408d-9f75-3439978aeb03 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:55.153634 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737160a-340e-408d-9f75-3439978aeb03 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:55.153906 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737160a-340e-408d-9f75-3439978aeb03 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:55.158335 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 220/443] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:55.159939 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9737160a-340e-408d-9f75-3439978aeb03 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:55.242771 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 224/444] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:08:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:08:55.248060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b0946-d267-42f4-8864-cc2f5026c5eb None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:55.248350 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b0946-d267-42f4-8864-cc2f5026c5eb None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:55.248634 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b0946-d267-42f4-8864-cc2f5026c5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:55.259925 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9737160a-340e-408d-9f75-3439978aeb03 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:55.266362 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-4e4b0946-d267-42f4-8864-cc2f5026c5eb None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:55.267725 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-4e4b0946-d267-42f4-8864-cc2f5026c5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:55.285768 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:55.293756 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4fb719-cadd-40d8-9ee5-71d30b72cc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:55.294525 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4fb719-cadd-40d8-9ee5-71d30b72cc21 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:55.294811 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4fb719-cadd-40d8-9ee5-71d30b72cc21 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:55.295087 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4fb719-cadd-40d8-9ee5-71d30b72cc21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:55.295876 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4fb719-cadd-40d8-9ee5-71d30b72cc21 None admin] RBAC: Adding query filter params (group.id=b4aab270fc5642f2b77d88507a8b6c2b, role.id=cdfc87279ea24b65bae3961035133725, scope.project.id=932a11c28fa2405793ce99d7097c63a5) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:55.296186 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4fb719-cadd-40d8-9ee5-71d30b72cc21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:55.297624 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4fb719-cadd-40d8-9ee5-71d30b72cc21 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:55.303816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 221/445] 10.222.0.46 () {56 vars in 1376 bytes} [Wed Jul 27 04:08:55 2022] GET /identity/v3/role_assignments?group.id=b4aab270fc5642f2b77d88507a8b6c2b&scope.project.id=932a11c28fa2405793ce99d7097c63a5&role.id=cdfc87279ea24b65bae3961035133725 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:55.304589 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4fb719-cadd-40d8-9ee5-71d30b72cc21 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:55.329104 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:55.338911 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 225/446] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:55.362501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b0946-d267-42f4-8864-cc2f5026c5eb None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:55.436303 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4fb719-cadd-40d8-9ee5-71d30b72cc21 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:55.478799 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b0946-d267-42f4-8864-cc2f5026c5eb None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:55.483231 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5965b672-267d-42be-899f-6a5c640a506e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:55.483887 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965b672-267d-42be-899f-6a5c640a506e None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:55.484168 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965b672-267d-42be-899f-6a5c640a506e None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:55.484447 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965b672-267d-42be-899f-6a5c640a506e None admin] PATH_INFO: `/v3/projects/5672fa915b944f15a9f6ebcb29dba655` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:55.485627 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5965b672-267d-42be-899f-6a5c640a506e None admin] RBAC: Authorizing `identity:get_project(project_id=5672fa915b944f15a9f6ebcb29dba655)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:55.487277 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5965b672-267d-42be-899f-6a5c640a506e None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:55.489285 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 222/447] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:55 2022] GET /identity/v3/projects/5672fa915b944f15a9f6ebcb29dba655 => generated 344 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:55.495297 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5965b672-267d-42be-899f-6a5c640a506e None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:55.518101 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f4767-3cb5-4016-95f5-602c079de9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:55.518761 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f4767-3cb5-4016-95f5-602c079de9a0 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:55.519042 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f4767-3cb5-4016-95f5-602c079de9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:55.519313 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f4767-3cb5-4016-95f5-602c079de9a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:55.529433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691f4767-3cb5-4016-95f5-602c079de9a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:08:55.534912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-691f4767-3cb5-4016-95f5-602c079de9a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:55.534912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:55.534912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:55.534912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:55.534912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:55.534912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:55.534912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:55.534912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:55.534912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:55.534912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:55.534912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:55.534912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:55.534912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:08:55.534912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:08:55.534912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:08:55.534912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:08:55.535918 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:08:55.535918 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:08:55.535918 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:08:55.535918 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:08:55.535918 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:55.535918 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:55.535918 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:55.535918 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:55.535918 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:55.535918 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:55.535918 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:55.535918 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:55.535918 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:55.535918 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:55.535918 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:55.535918 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:55.536932 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:55.536932 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:55.536932 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:55.536932 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:55.536932 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:08:55.536932 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:08:55.536932 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:08:55.536932 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:08:55.536932 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:08:55.536932 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:08:55.536932 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 04:08:55.536932 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:08:55.536932 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 226/448] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:08:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:08:55.541775 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f4767-3cb5-4016-95f5-602c079de9a0 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:55.558705 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:55.598479 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:55.601404 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5965b672-267d-42be-899f-6a5c640a506e None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:55.634474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353dcdd4-06f4-40e5-ba4b-86d887842d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:55.635063 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353dcdd4-06f4-40e5-ba4b-86d887842d05 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:55.635348 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353dcdd4-06f4-40e5-ba4b-86d887842d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:55.635629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353dcdd4-06f4-40e5-ba4b-86d887842d05 None admin] PATH_INFO: `/v3/groups/d1809a3517b443ea9d3bd07e68ee4be9` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:55.636796 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353dcdd4-06f4-40e5-ba4b-86d887842d05 None admin] RBAC: Authorizing `identity:get_group(group_id=d1809a3517b443ea9d3bd07e68ee4be9)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:55.638281 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353dcdd4-06f4-40e5-ba4b-86d887842d05 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:55.639880 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 223/449] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Jul 27 04:08:55 2022] GET /identity/v3/groups/d1809a3517b443ea9d3bd07e68ee4be9 => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:55.643535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f4767-3cb5-4016-95f5-602c079de9a0 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:55.646383 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353dcdd4-06f4-40e5-ba4b-86d887842d05 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:55.677527 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eea0f7-5752-46b7-95b1-9efddb2a634f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:55.678181 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eea0f7-5752-46b7-95b1-9efddb2a634f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:55.678470 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eea0f7-5752-46b7-95b1-9efddb2a634f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:55.678752 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eea0f7-5752-46b7-95b1-9efddb2a634f None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:55.679519 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eea0f7-5752-46b7-95b1-9efddb2a634f None admin] RBAC: Adding query filter params (name=member) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:55.679831 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eea0f7-5752-46b7-95b1-9efddb2a634f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:55.681226 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eea0f7-5752-46b7-95b1-9efddb2a634f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:55.689933 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 227/450] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 27 04:08:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:08:55.695465 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eea0f7-5752-46b7-95b1-9efddb2a634f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:55.744960 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353dcdd4-06f4-40e5-ba4b-86d887842d05 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:55.770883 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:55.779169 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ec9eaf-acca-487e-b845-192b40e5cb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:55.779795 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec9eaf-acca-487e-b845-192b40e5cb73 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:55.780074 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec9eaf-acca-487e-b845-192b40e5cb73 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:55.780345 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec9eaf-acca-487e-b845-192b40e5cb73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:55.781134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ec9eaf-acca-487e-b845-192b40e5cb73 None admin] RBAC: Adding query filter params (group.id=d1809a3517b443ea9d3bd07e68ee4be9, role.id=cf771a8e94ee46149eb081347fea6d67, scope.project.id=5672fa915b944f15a9f6ebcb29dba655) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:55.781441 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ec9eaf-acca-487e-b845-192b40e5cb73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:55.782928 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ec9eaf-acca-487e-b845-192b40e5cb73 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:55.788564 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 224/451] 10.222.0.46 () {56 vars in 1376 bytes} [Wed Jul 27 04:08:55 2022] GET /identity/v3/role_assignments?group.id=d1809a3517b443ea9d3bd07e68ee4be9&scope.project.id=5672fa915b944f15a9f6ebcb29dba655&role.id=cf771a8e94ee46149eb081347fea6d67 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:08:55.793883 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eea0f7-5752-46b7-95b1-9efddb2a634f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:55.815552 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:55.835052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d72323-45f7-432a-af43-64d6f4e63890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:55.835706 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d72323-45f7-432a-af43-64d6f4e63890 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:55.835989 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d72323-45f7-432a-af43-64d6f4e63890 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:55.836267 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d72323-45f7-432a-af43-64d6f4e63890 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:55.907362 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d72323-45f7-432a-af43-64d6f4e63890 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:55.909430 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d72323-45f7-432a-af43-64d6f4e63890 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:56.007316 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 228/452] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:08:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 04:08:56.012989 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ec9eaf-acca-487e-b845-192b40e5cb73 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:56.113336 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ec9eaf-acca-487e-b845-192b40e5cb73 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:56.148793 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea781974-bef4-4d9e-a267-64c434c41c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:56.149434 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea781974-bef4-4d9e-a267-64c434c41c3b None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:56.149721 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea781974-bef4-4d9e-a267-64c434c41c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:56.150026 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea781974-bef4-4d9e-a267-64c434c41c3b None admin] PATH_INFO: `/v3/projects/932a11c28fa2405793ce99d7097c63a5` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:56.151190 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea781974-bef4-4d9e-a267-64c434c41c3b None admin] RBAC: Authorizing `identity:get_project(project_id=932a11c28fa2405793ce99d7097c63a5)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:56.152632 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea781974-bef4-4d9e-a267-64c434c41c3b None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:56.154311 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 225/453] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:08:56 2022] GET /identity/v3/projects/932a11c28fa2405793ce99d7097c63a5 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:08:56.160035 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d72323-45f7-432a-af43-64d6f4e63890 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:56.178897 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:56.256064 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d72323-45f7-432a-af43-64d6f4e63890 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:56.288332 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37cb21-1fbc-49f8-b115-c88c1e2bcc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:56.288958 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37cb21-1fbc-49f8-b115-c88c1e2bcc47 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:56.289236 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37cb21-1fbc-49f8-b115-c88c1e2bcc47 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:56.289507 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37cb21-1fbc-49f8-b115-c88c1e2bcc47 None admin] PATH_INFO: `/v3/groups/b4aab270fc5642f2b77d88507a8b6c2b` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:56.290662 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c37cb21-1fbc-49f8-b115-c88c1e2bcc47 None admin] RBAC: Authorizing `identity:get_group(group_id=b4aab270fc5642f2b77d88507a8b6c2b)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:56.292081 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c37cb21-1fbc-49f8-b115-c88c1e2bcc47 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:56.293619 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 229/454] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Jul 27 04:08:56 2022] GET /identity/v3/groups/b4aab270fc5642f2b77d88507a8b6c2b => generated 226 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:56.299610 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea781974-bef4-4d9e-a267-64c434c41c3b None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:56.357528 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:56.398374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea781974-bef4-4d9e-a267-64c434c41c3b None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:56.430301 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c2cf7-33d7-4800-84f4-bbdd88904f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:56.430860 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c2cf7-33d7-4800-84f4-bbdd88904f03 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:56.431143 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c2cf7-33d7-4800-84f4-bbdd88904f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:56.431418 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c2cf7-33d7-4800-84f4-bbdd88904f03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:56.432215 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4c2cf7-33d7-4800-84f4-bbdd88904f03 None admin] RBAC: Adding query filter params (group.id=b4aab270fc5642f2b77d88507a8b6c2b, role.id=7b6f2ade82434ee79175385de17049eb, scope.project.id=932a11c28fa2405793ce99d7097c63a5) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:56.432529 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4c2cf7-33d7-4800-84f4-bbdd88904f03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:56.433959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4c2cf7-33d7-4800-84f4-bbdd88904f03 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:56.440556 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 226/455] 10.222.0.46 () {56 vars in 1376 bytes} [Wed Jul 27 04:08:56 2022] GET /identity/v3/role_assignments?group.id=b4aab270fc5642f2b77d88507a8b6c2b&scope.project.id=932a11c28fa2405793ce99d7097c63a5&role.id=7b6f2ade82434ee79175385de17049eb => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:08:59.080389 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c4ef2e-cd64-4eaa-84d9-142765677c71 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:59.080977 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c4ef2e-cd64-4eaa-84d9-142765677c71 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:59.080977 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c4ef2e-cd64-4eaa-84d9-142765677c71 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:59.082643 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 230/456] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:59.087933 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067731df-213e-4953-972d-37cef16e31db None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:59.088415 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067731df-213e-4953-972d-37cef16e31db None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:59.088495 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067731df-213e-4953-972d-37cef16e31db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:59.114932 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-067731df-213e-4953-972d-37cef16e31db None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:59.116151 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-067731df-213e-4953-972d-37cef16e31db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:59.203243 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 227/457] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 04:08:59.227820 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c4ef2e-cd64-4eaa-84d9-142765677c71 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:59.324248 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:59.347706 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c4ef2e-cd64-4eaa-84d9-142765677c71 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:59.391640 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c7b55-e6e4-4fa6-9f2a-afffff2eb660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:59.392434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c7b55-e6e4-4fa6-9f2a-afffff2eb660 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:59.392724 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c7b55-e6e4-4fa6-9f2a-afffff2eb660 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:59.392997 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c7b55-e6e4-4fa6-9f2a-afffff2eb660 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:59.413691 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8c7b55-e6e4-4fa6-9f2a-afffff2eb660 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:59.415185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8c7b55-e6e4-4fa6-9f2a-afffff2eb660 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:59.435205 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-7c8c7b55-e6e4-4fa6-9f2a-afffff2eb660 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:08:59.435205 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:08:59.435205 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:08:59.435205 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:08:59.435205 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:08:59.435205 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:08:59.435205 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:08:59.435205 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:08:59.435205 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:08:59.435205 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:08:59.435205 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:08:59.435205 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:08:59.435205 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:08:59.435205 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:08:59.435205 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:08:59.436102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:08:59.436102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:08:59.436102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:08:59.436102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:08:59.436102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:08:59.436102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:08:59.436102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:08:59.436102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:08:59.436102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:08:59.436102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:08:59.436102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:08:59.436102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:08:59.436102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:08:59.436102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:08:59.436102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:08:59.436102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:08:59.436102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:08:59.436944 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:08:59.436944 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:08:59.436944 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:08:59.436944 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:08:59.436944 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 231/458] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:08:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:08:59.443377 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067731df-213e-4953-972d-37cef16e31db None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:59.499907 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:59.546460 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067731df-213e-4953-972d-37cef16e31db None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:59.554613 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483b759-9258-4e37-b47d-442e52a86756 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:59.555107 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483b759-9258-4e37-b47d-442e52a86756 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:59.555536 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483b759-9258-4e37-b47d-442e52a86756 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:59.557040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 232/459] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 04:08:59.567795 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7ed93-e04c-4cd0-a0d2-4ee53a523e21 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:59.568380 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7ed93-e04c-4cd0-a0d2-4ee53a523e21 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:59.568809 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7ed93-e04c-4cd0-a0d2-4ee53a523e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:59.596754 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-8fd7ed93-e04c-4cd0-a0d2-4ee53a523e21 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:59.598394 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8fd7ed93-e04c-4cd0-a0d2-4ee53a523e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:59.627794 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d36db-9388-4945-a6a9-9cbe9cd7b73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:59.628504 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d36db-9388-4945-a6a9-9cbe9cd7b73d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:59.628790 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d36db-9388-4945-a6a9-9cbe9cd7b73d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:59.629233 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d36db-9388-4945-a6a9-9cbe9cd7b73d None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:59.629776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1d36db-9388-4945-a6a9-9cbe9cd7b73d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:08:59.630104 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1d36db-9388-4945-a6a9-9cbe9cd7b73d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:59.631564 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1d36db-9388-4945-a6a9-9cbe9cd7b73d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:59.672961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 228/460] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:08:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:08:59.678459 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d36db-9388-4945-a6a9-9cbe9cd7b73d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:59.710108 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 233/461] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:08:59.711843 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73328603-697e-458c-bfc1-2b81b6f05353 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:59.712308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73328603-697e-458c-bfc1-2b81b6f05353 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:59.712728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73328603-697e-458c-bfc1-2b81b6f05353 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:59.714562 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 234/462] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:59.720973 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31081b01-2a35-4421-abc1-4840d7bd870b None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:59.721499 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31081b01-2a35-4421-abc1-4840d7bd870b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:59.721955 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31081b01-2a35-4421-abc1-4840d7bd870b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:59.746492 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-31081b01-2a35-4421-abc1-4840d7bd870b None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:59.748275 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-31081b01-2a35-4421-abc1-4840d7bd870b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:59.804720 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d36db-9388-4945-a6a9-9cbe9cd7b73d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:08:59.821051 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:59.822002 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:08:59.834022 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 235/463] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 04:08:59.834022 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556582ee-439b-4816-a25e-464c248c92a4 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:59.834022 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556582ee-439b-4816-a25e-464c248c92a4 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:59.834022 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556582ee-439b-4816-a25e-464c248c92a4 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:59.840665 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 236/464] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:08:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:08:59.840665 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556582ee-439b-4816-a25e-464c248c92a4 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:08:59.843644 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a6ff6-0310-47e5-9f6d-1f5471dc842f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:08:59.844281 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a6ff6-0310-47e5-9f6d-1f5471dc842f None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:59.844568 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a6ff6-0310-47e5-9f6d-1f5471dc842f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:59.844845 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a6ff6-0310-47e5-9f6d-1f5471dc842f None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:59.845601 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122a6ff6-0310-47e5-9f6d-1f5471dc842f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:08:59.847066 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122a6ff6-0310-47e5-9f6d-1f5471dc842f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:08:59.872536 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 229/465] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 27 04:08:59 2022] POST /identity/v3/users => generated 246 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:08:59.873748 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf49a0-e009-46bf-a88a-4747947af8e6 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:08:59.874190 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf49a0-e009-46bf-a88a-4747947af8e6 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:08:59.874481 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf49a0-e009-46bf-a88a-4747947af8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:08:59.898266 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-3bcf49a0-e009-46bf-a88a-4747947af8e6 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:08:59.900118 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3bcf49a0-e009-46bf-a88a-4747947af8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:08:59.966856 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556582ee-439b-4816-a25e-464c248c92a4 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:00.006141 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 230/466] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:08:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:09:00.007633 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf49a0-e009-46bf-a88a-4747947af8e6 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:00.009490 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc86e12-ddef-4aca-946d-47835b682ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:00.010443 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc86e12-ddef-4aca-946d-47835b682ebf None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:00.010978 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc86e12-ddef-4aca-946d-47835b682ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:00.011449 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc86e12-ddef-4aca-946d-47835b682ebf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:00.036978 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc86e12-ddef-4aca-946d-47835b682ebf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:00.038520 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc86e12-ddef-4aca-946d-47835b682ebf None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:00.056912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-0bc86e12-ddef-4aca-946d-47835b682ebf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:00.056912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:00.056912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:00.056912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:00.056912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:00.056912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:00.056912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:00.056912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:00.056912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:00.056912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:00.056912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:00.056912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:00.056912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:00.056912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:00.056912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:00.057848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:00.057848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:00.057848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:00.057848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:00.057848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:00.057848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:00.057848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:00.057848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:00.057848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:00.057848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:00.057848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:00.057848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:00.057848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:00.057848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:00.057848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:00.057848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:00.057848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:00.058731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:00.058731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:00.058731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:00.058731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:00.062047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 237/467] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:08:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:00.063348 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc86e12-ddef-4aca-946d-47835b682ebf None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:00.129309 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:00.168409 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:00.171403 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf49a0-e009-46bf-a88a-4747947af8e6 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:00.190093 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc86e12-ddef-4aca-946d-47835b682ebf None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:00.219178 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b8432-af8a-47bb-a45c-14758c5195f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:00.220114 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b8432-af8a-47bb-a45c-14758c5195f8 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:00.221706 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b8432-af8a-47bb-a45c-14758c5195f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:00.222240 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b8432-af8a-47bb-a45c-14758c5195f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:00.226584 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcbaf03-d6fb-4e84-913e-469259770e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:00.227352 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbaf03-d6fb-4e84-913e-469259770e9e None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:00.227727 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbaf03-d6fb-4e84-913e-469259770e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:00.228119 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbaf03-d6fb-4e84-913e-469259770e9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:00.234934 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b8432-af8a-47bb-a45c-14758c5195f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:00.238190 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b8432-af8a-47bb-a45c-14758c5195f8 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:00.240871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcbaf03-d6fb-4e84-913e-469259770e9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:00.242559 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcbaf03-d6fb-4e84-913e-469259770e9e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:00.257724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-565b8432-af8a-47bb-a45c-14758c5195f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:00.257724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:00.257724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:00.257724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:00.257724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:00.257724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:00.257724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:00.257724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:00.257724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:00.257724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:00.257724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:00.257724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:00.257724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:00.257724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:00.257724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:00.258815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:00.258815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:00.258815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:00.258815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:00.258815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:00.258815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:00.258815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:00.258815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:00.258815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:00.258815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:00.258815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:00.258815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:00.258815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:00.258815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:00.258815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:00.258815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:00.258815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:00.259857 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:00.259857 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:00.259857 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:00.259857 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:00.260501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-cfcbaf03-d6fb-4e84-913e-469259770e9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:00.260501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:00.260501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:00.260501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:00.260501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:00.260501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:00.260501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:00.260501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:00.260501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:00.260501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:00.260501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:00.260501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:00.260501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:00.260501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:00.260501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:00.261476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:00.261476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:00.261476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:00.261476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:00.261476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:00.261476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:00.261476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:00.261476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:00.261476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:00.261476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:00.261476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:00.261476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:00.261476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:00.261476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:00.261476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:00.261476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:00.261476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:00.262646 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:00.262646 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:00.262646 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:00.262646 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:00.264098 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 231/469] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:00.265082 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b8432-af8a-47bb-a45c-14758c5195f8 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:00.268543 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 238/469] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:00.270359 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcbaf03-d6fb-4e84-913e-469259770e9e None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:00.397825 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcbaf03-d6fb-4e84-913e-469259770e9e None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:00.400275 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:00.402385 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b8432-af8a-47bb-a45c-14758c5195f8 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:00.440041 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0ed2c-f15d-4c14-802e-1b1648931bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:00.440834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0ed2c-f15d-4c14-802e-1b1648931bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:00.441216 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0ed2c-f15d-4c14-802e-1b1648931bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:00.441591 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0ed2c-f15d-4c14-802e-1b1648931bd3 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:00.442582 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0ed2c-f15d-4c14-802e-1b1648931bd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:00.443031 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0ed2c-f15d-4c14-802e-1b1648931bd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:00.444321 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774934bd-6b49-415e-94a5-9d56a23cd08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:00.444575 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0ed2c-f15d-4c14-802e-1b1648931bd3 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:00.445258 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774934bd-6b49-415e-94a5-9d56a23cd08b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:00.445680 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774934bd-6b49-415e-94a5-9d56a23cd08b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:00.446125 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774934bd-6b49-415e-94a5-9d56a23cd08b None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:00.447193 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774934bd-6b49-415e-94a5-9d56a23cd08b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:00.448139 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774934bd-6b49-415e-94a5-9d56a23cd08b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:00.449648 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774934bd-6b49-415e-94a5-9d56a23cd08b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:00.459932 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 232/470] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:00.460667 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0ed2c-f15d-4c14-802e-1b1648931bd3 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:00.465860 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 239/471] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:00.469375 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774934bd-6b49-415e-94a5-9d56a23cd08b None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:00.484167 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:00.583672 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774934bd-6b49-415e-94a5-9d56a23cd08b None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:00.600192 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0ed2c-f15d-4c14-802e-1b1648931bd3 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:00.619177 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d5f41c-a9a1-46b4-8893-ae3da22e303f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:00.619791 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5f41c-a9a1-46b4-8893-ae3da22e303f None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:00.620071 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5f41c-a9a1-46b4-8893-ae3da22e303f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:00.620342 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5f41c-a9a1-46b4-8893-ae3da22e303f None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:00.621028 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d5f41c-a9a1-46b4-8893-ae3da22e303f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:00.622510 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d5f41c-a9a1-46b4-8893-ae3da22e303f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:00.645528 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895a7e3-76b8-4f7d-93a9-fa3c76b0de8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:00.646563 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895a7e3-76b8-4f7d-93a9-fa3c76b0de8a None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:00.647104 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895a7e3-76b8-4f7d-93a9-fa3c76b0de8a None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:00.647599 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895a7e3-76b8-4f7d-93a9-fa3c76b0de8a None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:00.648562 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9895a7e3-76b8-4f7d-93a9-fa3c76b0de8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:00.649083 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9895a7e3-76b8-4f7d-93a9-fa3c76b0de8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:00.650738 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9895a7e3-76b8-4f7d-93a9-fa3c76b0de8a None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:00.652927 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 240/472] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 27 04:09:00 2022] POST /identity/v3/users => generated 248 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:09:00.654322 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d5f41c-a9a1-46b4-8893-ae3da22e303f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:00.666243 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 233/473] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:00.668044 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b3286-a765-400f-9c80-55ee0bf67c62 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:00.668556 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b3286-a765-400f-9c80-55ee0bf67c62 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:00.669041 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b3286-a765-400f-9c80-55ee0bf67c62 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:00.671070 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 234/474] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:00.684464 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:00.685789 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fc590-49dc-4e27-8c1d-ea677a4a6bf1 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:00.685789 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fc590-49dc-4e27-8c1d-ea677a4a6bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:00.685789 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fc590-49dc-4e27-8c1d-ea677a4a6bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:00.718767 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-590fc590-49dc-4e27-8c1d-ea677a4a6bf1 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:00.720408 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-590fc590-49dc-4e27-8c1d-ea677a4a6bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:00.784525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d5f41c-a9a1-46b4-8893-ae3da22e303f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:00.824172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 235/475] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:09:00.826069 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590fc590-49dc-4e27-8c1d-ea677a4a6bf1 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:00.830444 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f631bb-4174-4476-bfee-4cb3f13be17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:00.831129 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f631bb-4174-4476-bfee-4cb3f13be17d None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:00.831416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f631bb-4174-4476-bfee-4cb3f13be17d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:00.831698 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f631bb-4174-4476-bfee-4cb3f13be17d None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:00.832391 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f631bb-4174-4476-bfee-4cb3f13be17d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:00.833859 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f631bb-4174-4476-bfee-4cb3f13be17d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:00.847754 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:00.864352 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 241/476] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 27 04:09:00 2022] POST /identity/v3/users => generated 249 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:09:00.866030 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f631bb-4174-4476-bfee-4cb3f13be17d None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:00.966704 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590fc590-49dc-4e27-8c1d-ea677a4a6bf1 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:00.988268 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:01.024894 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8dc6c5-df7b-4962-941b-1e83c894122f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:01.026109 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8dc6c5-df7b-4962-941b-1e83c894122f None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:01.026625 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8dc6c5-df7b-4962-941b-1e83c894122f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:01.027104 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8dc6c5-df7b-4962-941b-1e83c894122f None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:01.028031 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8dc6c5-df7b-4962-941b-1e83c894122f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:01.029483 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8dc6c5-df7b-4962-941b-1e83c894122f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:01.036755 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f631bb-4174-4476-bfee-4cb3f13be17d None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:01.063803 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 236/477] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 27 04:09:00 2022] POST /identity/v3/users => generated 248 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:09:01.083040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e533ea15-cfd0-4a14-a77a-d3ccc858d256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:01.083708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533ea15-cfd0-4a14-a77a-d3ccc858d256 None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:01.083998 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533ea15-cfd0-4a14-a77a-d3ccc858d256 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:01.084304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533ea15-cfd0-4a14-a77a-d3ccc858d256 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:01.085045 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e533ea15-cfd0-4a14-a77a-d3ccc858d256 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:01.086597 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e533ea15-cfd0-4a14-a77a-d3ccc858d256 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:01.095884 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.sql.core [None req-e533ea15-cfd0-4a14-a77a-d3ccc858d256 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 27 04:09:01.095884 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 27 04:09:01.095884 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [parameters: {'id': '70ffd00960f44b3ba174fca5fd8ae468', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 27 04:09:01.095884 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80239) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 27 04:09:01.113484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-e533ea15-cfd0-4a14-a77a-d3ccc858d256 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 27 04:09:01.113484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:01.113484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 04:09:01.113484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 04:09:01.113484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 04:09:01.113484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 04:09:01.113484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 04:09:01.113484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 04:09:01.113484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 04:09:01.113484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application conn.query(q) Jul 27 04:09:01.113484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 04:09:01.113484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 04:09:01.113484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 04:09:01.113484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application result.read() Jul 27 04:09:01.113484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 04:09:01.114535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 04:09:01.114535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 04:09:01.114535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 04:09:01.114535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 04:09:01.114535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 04:09:01.114535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 04:09:01.114535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 04:09:01.114535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 27 04:09:01.114535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:01.114535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 27 04:09:01.114535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:01.114535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:01.114535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 27 04:09:01.114535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 27 04:09:01.114535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 27 04:09:01.114535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return ref.to_dict() Jul 27 04:09:01.114535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 04:09:01.114535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application next(self.gen) Jul 27 04:09:01.115447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 27 04:09:01.115447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application yield resource Jul 27 04:09:01.115447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 04:09:01.115447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application next(self.gen) Jul 27 04:09:01.115447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 27 04:09:01.115447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self.session.rollback() Jul 27 04:09:01.115447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 27 04:09:01.115447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self.force_reraise() Jul 27 04:09:01.115447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 27 04:09:01.115447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise self.value Jul 27 04:09:01.115447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 27 04:09:01.115447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 27 04:09:01.115447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 27 04:09:01.115447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application session.commit() Jul 27 04:09:01.115447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 27 04:09:01.115447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 27 04:09:01.115447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 27 04:09:01.116349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._prepare_impl() Jul 27 04:09:01.116349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 27 04:09:01.116349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self.session.flush() Jul 27 04:09:01.116349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 27 04:09:01.116349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._flush(objects) Jul 27 04:09:01.116349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 27 04:09:01.116349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 27 04:09:01.116349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 27 04:09:01.116349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application compat.raise_( Jul 27 04:09:01.116349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 04:09:01.116349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception Jul 27 04:09:01.116349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 27 04:09:01.116349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application flush_context.execute() Jul 27 04:09:01.116349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 27 04:09:01.116349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rec.execute(self) Jul 27 04:09:01.116349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 27 04:09:01.116349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 27 04:09:01.117208 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 27 04:09:01.117208 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 27 04:09:01.117208 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 27 04:09:01.117208 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 27 04:09:01.117208 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 27 04:09:01.117208 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 27 04:09:01.117208 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 27 04:09:01.117208 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 27 04:09:01.117208 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 27 04:09:01.117208 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 27 04:09:01.117208 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 27 04:09:01.117208 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 27 04:09:01.117208 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 27 04:09:01.117208 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 27 04:09:01.117208 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 04:09:01.117208 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception Jul 27 04:09:01.119526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 04:09:01.119526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 04:09:01.119526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 04:09:01.119526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 04:09:01.119526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 04:09:01.119526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 04:09:01.119526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 04:09:01.119526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application conn.query(q) Jul 27 04:09:01.119526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 04:09:01.119526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 04:09:01.119526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 04:09:01.119526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application result.read() Jul 27 04:09:01.119526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 04:09:01.119526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 04:09:01.119526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 04:09:01.119526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 04:09:01.121559 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 04:09:01.121559 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 04:09:01.121559 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 04:09:01.121559 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 04:09:01.121559 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 27 04:09:01.121559 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 27 04:09:01.121559 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [parameters: {'id': '70ffd00960f44b3ba174fca5fd8ae468', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 27 04:09:01.121559 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 27 04:09:01.121559 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:01.121559 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 04:09:01.121559 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:01.121559 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:01.121559 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:01.121559 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:01.121559 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:01.121559 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:01.128511 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:01.128511 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:01.128511 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:01.128511 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:01.128511 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:01.128511 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:01.128511 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 27 04:09:01.128511 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 27 04:09:01.128511 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:01.128511 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:01.128511 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 27 04:09:01.128511 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 27 04:09:01.128511 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 27 04:09:01.128511 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 27 04:09:01.128511 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 27 04:09:01.128511 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:01.131074 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 242/478] 10.222.0.46 () {60 vars in 1131 bytes} [Wed Jul 27 04:09:00 2022] POST /identity/v3/roles => generated 144 bytes in 250 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:01.131212 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8dc6c5-df7b-4962-941b-1e83c894122f None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:01.202145 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:01.255203 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8dc6c5-df7b-4962-941b-1e83c894122f None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:01.295071 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced94a7-cda7-437c-9a49-b69eb98e9f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:01.295730 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced94a7-cda7-437c-9a49-b69eb98e9f98 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:01.296012 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced94a7-cda7-437c-9a49-b69eb98e9f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:01.296289 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced94a7-cda7-437c-9a49-b69eb98e9f98 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:01.308102 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced94a7-cda7-437c-9a49-b69eb98e9f98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:09:01.314237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-1ced94a7-cda7-437c-9a49-b69eb98e9f98 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 04:09:01.314237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:01.314237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:01.314237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:01.314237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:01.314237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:01.314237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:01.314237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:01.314237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:01.314237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:01.314237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:01.314237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:01.314237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:09:01.314237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:09:01.314237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:09:01.314237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:09:01.316496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:09:01.316496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:09:01.316496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:09:01.316496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:09:01.316496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:01.316496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:01.316496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:01.316496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:01.316496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:01.316496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:01.316496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:01.316496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:01.316496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:01.316496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:01.316496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:01.316496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:01.318436 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:01.318436 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:01.318436 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:01.318436 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:01.318436 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:09:01.318436 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:09:01.318436 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:09:01.318436 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:09:01.318436 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:09:01.318436 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:09:01.318436 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 04:09:01.318436 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:01.318436 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 237/479] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 27 04:09:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:09:01.326109 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e533ea15-cfd0-4a14-a77a-d3ccc858d256 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:01.439831 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e533ea15-cfd0-4a14-a77a-d3ccc858d256 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:01.476298 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd86e3b-31a4-4350-b9ba-288615ad7bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:01.476932 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd86e3b-31a4-4350-b9ba-288615ad7bad None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:01.477225 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd86e3b-31a4-4350-b9ba-288615ad7bad None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:01.477505 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd86e3b-31a4-4350-b9ba-288615ad7bad None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:01.478276 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd86e3b-31a4-4350-b9ba-288615ad7bad None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:01.478588 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd86e3b-31a4-4350-b9ba-288615ad7bad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:01.479970 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd86e3b-31a4-4350-b9ba-288615ad7bad None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:01.489434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:01.489685 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 243/480] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:09:02.138069 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70454859-9f70-4711-b27f-b1307da629e9 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:02.138349 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70454859-9f70-4711-b27f-b1307da629e9 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:02.138621 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70454859-9f70-4711-b27f-b1307da629e9 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:02.140231 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 238/481] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:02.145542 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17e6ac-ac53-4551-ba5d-b2a6754ad1ce None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:02.146023 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17e6ac-ac53-4551-ba5d-b2a6754ad1ce None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:02.146115 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17e6ac-ac53-4551-ba5d-b2a6754ad1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:02.171417 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-af17e6ac-ac53-4551-ba5d-b2a6754ad1ce None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:02.172618 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-af17e6ac-ac53-4551-ba5d-b2a6754ad1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:02.265623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 244/482] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:09:02.300166 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70454859-9f70-4711-b27f-b1307da629e9 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:02.415547 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70454859-9f70-4711-b27f-b1307da629e9 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:02.444428 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:02.466789 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c3938-cf5f-45ff-a8ce-498b01f7a26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:02.470023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c3938-cf5f-45ff-a8ce-498b01f7a26a None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:02.470023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c3938-cf5f-45ff-a8ce-498b01f7a26a None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:02.470023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c3938-cf5f-45ff-a8ce-498b01f7a26a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:02.488729 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c3938-cf5f-45ff-a8ce-498b01f7a26a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:09:02.493643 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-c21c3938-cf5f-45ff-a8ce-498b01f7a26a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:02.493643 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:02.493643 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:02.493643 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:02.493643 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:02.493643 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:02.493643 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:02.493643 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:02.493643 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:02.493643 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:02.493643 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:02.493643 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:02.493643 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:09:02.493643 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:09:02.493643 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:09:02.493643 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:09:02.494555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:09:02.494555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:09:02.494555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:09:02.494555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:09:02.494555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:02.494555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:02.494555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:02.494555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:02.494555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:02.494555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:02.494555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:02.494555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:02.494555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:02.494555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:02.494555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:02.494555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:02.495339 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:02.495339 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:02.495339 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:02.495339 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:02.495339 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:09:02.495339 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:09:02.495339 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:09:02.495339 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:09:02.495339 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:09:02.495339 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:09:02.495339 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:02.495339 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:02.499296 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 239/483] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 27 04:09:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:02.506311 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17e6ac-ac53-4551-ba5d-b2a6754ad1ce None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:02.656624 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17e6ac-ac53-4551-ba5d-b2a6754ad1ce None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:02.710210 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:02.717386 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c9868-569b-4778-a722-46eaf3c49bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:02.718206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c9868-569b-4778-a722-46eaf3c49bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:02.718485 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c9868-569b-4778-a722-46eaf3c49bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:02.718755 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c9868-569b-4778-a722-46eaf3c49bc3 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:02.719487 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1c9868-569b-4778-a722-46eaf3c49bc3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:02.719785 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1c9868-569b-4778-a722-46eaf3c49bc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:02.721405 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1c9868-569b-4778-a722-46eaf3c49bc3 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:02.730152 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 245/484] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 27 04:09:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:02.736244 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c3938-cf5f-45ff-a8ce-498b01f7a26a None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:02.853500 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c3938-cf5f-45ff-a8ce-498b01f7a26a None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:02.893653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502be6ef-dd28-4724-93e4-dff9f5c3ffd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:02.894341 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502be6ef-dd28-4724-93e4-dff9f5c3ffd8 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:02.894624 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502be6ef-dd28-4724-93e4-dff9f5c3ffd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:02.894901 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502be6ef-dd28-4724-93e4-dff9f5c3ffd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:02.909521 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502be6ef-dd28-4724-93e4-dff9f5c3ffd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:02.911031 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502be6ef-dd28-4724-93e4-dff9f5c3ffd8 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:02.932982 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-502be6ef-dd28-4724-93e4-dff9f5c3ffd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:02.932982 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:02.932982 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:02.932982 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:02.932982 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:02.932982 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:02.932982 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:02.932982 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:02.932982 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:02.932982 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:02.932982 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:02.932982 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:02.932982 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:02.932982 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:02.932982 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:02.935006 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:02.935006 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:02.935006 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:02.935006 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:02.935006 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:02.935006 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:02.935006 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:02.935006 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:02.935006 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:02.935006 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:02.935006 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:02.935006 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:02.935006 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:02.935006 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:02.935006 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:02.935006 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:02.935006 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:02.937077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:02.937077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:02.937077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:02.937077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:02.939368 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 240/485] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:02.947155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c9868-569b-4778-a722-46eaf3c49bc3 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:03.073811 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c9868-569b-4778-a722-46eaf3c49bc3 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:03.109577 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1c7f7-89f7-45bc-a3ff-00515d1c1514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:03.110248 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1c7f7-89f7-45bc-a3ff-00515d1c1514 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:03.110574 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1c7f7-89f7-45bc-a3ff-00515d1c1514 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:03.111009 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1c7f7-89f7-45bc-a3ff-00515d1c1514 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:03.111872 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1c7f7-89f7-45bc-a3ff-00515d1c1514 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:03.112191 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1c7f7-89f7-45bc-a3ff-00515d1c1514 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:03.113588 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1c7f7-89f7-45bc-a3ff-00515d1c1514 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:03.132046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 246/486] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:03.142793 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502be6ef-dd28-4724-93e4-dff9f5c3ffd8 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:03.195552 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:03.284850 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502be6ef-dd28-4724-93e4-dff9f5c3ffd8 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:03.335277 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f624459-d567-458d-80fc-6137c92456a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:03.336050 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f624459-d567-458d-80fc-6137c92456a9 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:03.336222 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f624459-d567-458d-80fc-6137c92456a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:03.336471 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f624459-d567-458d-80fc-6137c92456a9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:03.344622 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f624459-d567-458d-80fc-6137c92456a9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:03.346736 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f624459-d567-458d-80fc-6137c92456a9 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:03.356771 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde73fd-7af2-47cd-a0aa-d353194aa3c5 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:03.357313 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde73fd-7af2-47cd-a0aa-d353194aa3c5 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:03.357313 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde73fd-7af2-47cd-a0aa-d353194aa3c5 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:03.358850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 247/487] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:03.363457 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-9f624459-d567-458d-80fc-6137c92456a9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 04:09:03.363457 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:03.363457 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:03.363457 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:03.363457 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:03.363457 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:03.363457 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:03.363457 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:03.363457 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:03.363457 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:03.363457 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:03.363457 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:03.363457 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:09:03.363457 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:09:03.363457 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:09:03.363457 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:09:03.365524 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:03.365524 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:03.365524 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:09:03.365524 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:03.365524 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:09:03.365524 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:03.365524 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:03.365524 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:03.365524 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:03.365524 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:03.365524 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:03.365524 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:03.365524 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:03.365524 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:03.365524 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:03.365524 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:03.365524 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:03.365524 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:03.367801 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:03.367801 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:03.367801 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:09:03.367801 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:09:03.367801 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:09:03.367801 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:09:03.367801 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:09:03.367801 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:09:03.367801 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 04:09:03.367801 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:03.367801 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 241/488] 10.222.0.46 () {56 vars in 1093 bytes} [Wed Jul 27 04:09:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:03.373791 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2802068-c706-4e97-9929-bd68db9f0eb9 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:03.373791 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2802068-c706-4e97-9929-bd68db9f0eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:03.373791 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2802068-c706-4e97-9929-bd68db9f0eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:03.374456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f624459-d567-458d-80fc-6137c92456a9 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:03.409949 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-d2802068-c706-4e97-9929-bd68db9f0eb9 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:03.411196 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d2802068-c706-4e97-9929-bd68db9f0eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:03.473941 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:03.492328 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f624459-d567-458d-80fc-6137c92456a9 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:03.530120 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:03.566345 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca495d-cb14-424f-b0f4-c6e69b6ea53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:03.566982 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca495d-cb14-424f-b0f4-c6e69b6ea53c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:03.567266 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca495d-cb14-424f-b0f4-c6e69b6ea53c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:03.567542 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca495d-cb14-424f-b0f4-c6e69b6ea53c None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:03.569075 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ca495d-cb14-424f-b0f4-c6e69b6ea53c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:03.569387 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ca495d-cb14-424f-b0f4-c6e69b6ea53c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:03.570865 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ca495d-cb14-424f-b0f4-c6e69b6ea53c None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:03.578079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 248/489] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:09:03.634349 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 242/490] 10.222.0.46 () {56 vars in 1133 bytes} [Wed Jul 27 04:09:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:09:03.640336 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2802068-c706-4e97-9929-bd68db9f0eb9 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:03.640560 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca495d-cb14-424f-b0f4-c6e69b6ea53c None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:03.768020 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2802068-c706-4e97-9929-bd68db9f0eb9 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:03.773820 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca495d-cb14-424f-b0f4-c6e69b6ea53c None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:03.806911 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1dea69-afb8-465a-ac0b-db89d1cc6b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:03.807663 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1dea69-afb8-465a-ac0b-db89d1cc6b45 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:03.808074 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1dea69-afb8-465a-ac0b-db89d1cc6b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:03.808376 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1dea69-afb8-465a-ac0b-db89d1cc6b45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:03.824977 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad2587c-a2d5-46a1-9eef-f09e5a88481c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:03.826079 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2587c-a2d5-46a1-9eef-f09e5a88481c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:03.826543 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2587c-a2d5-46a1-9eef-f09e5a88481c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:03.827157 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2587c-a2d5-46a1-9eef-f09e5a88481c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:03.840645 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad2587c-a2d5-46a1-9eef-f09e5a88481c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:09:03.848297 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-fad2587c-a2d5-46a1-9eef-f09e5a88481c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:03.848297 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:03.848297 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:03.848297 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:03.848297 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:03.848297 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:03.848297 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:03.848297 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:03.848297 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:03.848297 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:03.848297 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:03.848297 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:03.848297 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:09:03.848297 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:09:03.848297 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:09:03.848297 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:09:03.849317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:09:03.849317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:09:03.849317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:09:03.849317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:09:03.849317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:03.849317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:03.849317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:03.849317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:03.849317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:03.849317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:03.849317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:03.849317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:03.849317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:03.849317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:03.849317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:03.849317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:03.850142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:03.850142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:03.850142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:03.850142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:03.850142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:09:03.850142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:09:03.850142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:09:03.850142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:09:03.850142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:09:03.850142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:09:03.850142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:03.850142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:03.853310 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 243/491] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 27 04:09:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:03.854725 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c104440-35d0-4b02-b81c-8c8922b382cb None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:03.855166 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c104440-35d0-4b02-b81c-8c8922b382cb None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:03.855589 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c104440-35d0-4b02-b81c-8c8922b382cb None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:03.861243 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 244/492] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:03.862334 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c104440-35d0-4b02-b81c-8c8922b382cb None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:03.875252 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:03.878302 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:03.903731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1dea69-afb8-465a-ac0b-db89d1cc6b45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:03.905260 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1dea69-afb8-465a-ac0b-db89d1cc6b45 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:03.983925 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c104440-35d0-4b02-b81c-8c8922b382cb None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:04.014009 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 249/493] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:09:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:09:04.014784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425e32a-82e7-4b20-94d1-e6fb0e4b5d38 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:04.015072 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425e32a-82e7-4b20-94d1-e6fb0e4b5d38 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:04.015345 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425e32a-82e7-4b20-94d1-e6fb0e4b5d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:04.026899 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a6616-7e28-4cbf-a42b-6061085e8f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:04.027550 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a6616-7e28-4cbf-a42b-6061085e8f60 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:04.028224 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a6616-7e28-4cbf-a42b-6061085e8f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:04.028517 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a6616-7e28-4cbf-a42b-6061085e8f60 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:04.029250 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a6616-7e28-4cbf-a42b-6061085e8f60 None admin] RBAC: Adding query filter params (name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:04.029556 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a6616-7e28-4cbf-a42b-6061085e8f60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:04.031135 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a6616-7e28-4cbf-a42b-6061085e8f60 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:04.034068 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-3425e32a-82e7-4b20-94d1-e6fb0e4b5d38 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:04.035240 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-3425e32a-82e7-4b20-94d1-e6fb0e4b5d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:04.044697 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 245/494] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 27 04:09:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:04.044697 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b85b7-40ad-4526-b9d0-59b3213604d4 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:04.044697 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b85b7-40ad-4526-b9d0-59b3213604d4 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:04.044697 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b85b7-40ad-4526-b9d0-59b3213604d4 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:04.044697 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 246/495] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:04.044697 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774b85b7-40ad-4526-b9d0-59b3213604d4 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:04.117996 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 250/496] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:09:04.117996 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425e32a-82e7-4b20-94d1-e6fb0e4b5d38 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:04.131883 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:04.133062 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:04.173303 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774b85b7-40ad-4526-b9d0-59b3213604d4 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:04.211616 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f3b4d-8180-4199-8412-0a10f055294b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:04.212269 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f3b4d-8180-4199-8412-0a10f055294b None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:04.212616 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f3b4d-8180-4199-8412-0a10f055294b None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:04.212816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f3b4d-8180-4199-8412-0a10f055294b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:04.224670 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848f3b4d-8180-4199-8412-0a10f055294b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:04.226150 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848f3b4d-8180-4199-8412-0a10f055294b None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:04.232050 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425e32a-82e7-4b20-94d1-e6fb0e4b5d38 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:04.242516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-848f3b4d-8180-4199-8412-0a10f055294b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:04.242516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:04.242516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:04.242516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:04.242516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:04.242516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:04.242516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:04.242516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:04.242516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:04.242516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:04.242516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:04.242516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:04.242516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:04.242516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:04.242516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:04.243384 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:04.243384 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:04.243384 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:04.243384 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:04.243384 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:04.243384 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:04.243384 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:04.243384 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:04.243384 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:04.243384 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:04.243384 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:04.243384 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:04.243384 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:04.243384 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:04.243384 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:04.243384 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:04.243384 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:04.244204 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:04.244204 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:04.244204 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:04.244204 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:04.244868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 247/497] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:04.245766 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba8439-dbdc-41f9-93d9-a1a2dec5bd4b None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:04.246079 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba8439-dbdc-41f9-93d9-a1a2dec5bd4b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:04.246353 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba8439-dbdc-41f9-93d9-a1a2dec5bd4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:04.264951 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-02ba8439-dbdc-41f9-93d9-a1a2dec5bd4b None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:04.266168 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-02ba8439-dbdc-41f9-93d9-a1a2dec5bd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:04.270711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b91c5e-213e-4a3e-9d13-0ddd3653ee3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:04.271359 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b91c5e-213e-4a3e-9d13-0ddd3653ee3e None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:04.271658 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b91c5e-213e-4a3e-9d13-0ddd3653ee3e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:04.271917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b91c5e-213e-4a3e-9d13-0ddd3653ee3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:04.281705 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b91c5e-213e-4a3e-9d13-0ddd3653ee3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:04.283856 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b91c5e-213e-4a3e-9d13-0ddd3653ee3e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:04.298184 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-59b91c5e-213e-4a3e-9d13-0ddd3653ee3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:04.298184 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:04.298184 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:04.298184 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:04.298184 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:04.298184 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:04.298184 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:04.298184 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:04.298184 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:04.298184 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:04.298184 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:04.298184 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:04.298184 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:04.298184 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:04.298184 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:04.299508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:04.299508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:04.299508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:04.299508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:04.299508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:04.299508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:04.299508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:04.299508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:04.299508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:04.299508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:04.299508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:04.299508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:04.299508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:04.299508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:04.299508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:04.299508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:04.299508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:04.300354 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:04.300354 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:04.300354 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:04.300354 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:04.300354 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 251/498] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:04.300354 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b91c5e-213e-4a3e-9d13-0ddd3653ee3e None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:04.337044 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 248/499] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:09:04.338876 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b37fb-3d36-4964-aead-ec8670e142f9 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:04.339164 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b37fb-3d36-4964-aead-ec8670e142f9 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:04.339442 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b37fb-3d36-4964-aead-ec8670e142f9 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:04.340692 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 249/500] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:04.341381 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b37fb-3d36-4964-aead-ec8670e142f9 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:04.365940 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:04.404091 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:04.405385 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b91c5e-213e-4a3e-9d13-0ddd3653ee3e None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:04.440526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fda6fa-5fa7-4496-9d58-0b6cb8403d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:04.441098 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fda6fa-5fa7-4496-9d58-0b6cb8403d38 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:04.441378 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fda6fa-5fa7-4496-9d58-0b6cb8403d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:04.441652 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fda6fa-5fa7-4496-9d58-0b6cb8403d38 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:04.451344 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fda6fa-5fa7-4496-9d58-0b6cb8403d38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:09:04.455375 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b37fb-3d36-4964-aead-ec8670e142f9 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:04.457198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-f9fda6fa-5fa7-4496-9d58-0b6cb8403d38 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:04.457198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:04.457198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:04.457198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:04.457198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:04.457198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:04.457198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:04.457198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:04.457198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:04.457198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:04.457198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:04.457198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:04.457198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:09:04.457198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:09:04.457198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:09:04.457198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:09:04.458154 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:09:04.458154 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:09:04.458154 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:09:04.458154 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:09:04.458154 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:04.458154 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:04.458154 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:04.458154 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:04.458154 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:04.458154 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:04.458154 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:04.458154 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:04.458154 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:04.458154 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:04.458154 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:04.458154 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:04.460044 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:04.460044 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:04.460044 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:04.460044 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:04.460044 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:09:04.460044 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:09:04.460044 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:09:04.460044 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:09:04.460044 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:09:04.460044 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:09:04.460044 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:04.460044 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:04.460044 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 252/501] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 27 04:09:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:04.460044 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fda6fa-5fa7-4496-9d58-0b6cb8403d38 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:04.491882 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01afe5c5-c6e5-4862-bc25-c9b23a4e458f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:04.492516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afe5c5-c6e5-4862-bc25-c9b23a4e458f None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:04.492793 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afe5c5-c6e5-4862-bc25-c9b23a4e458f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:04.493091 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afe5c5-c6e5-4862-bc25-c9b23a4e458f None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:04.493791 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01afe5c5-c6e5-4862-bc25-c9b23a4e458f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:04.494131 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01afe5c5-c6e5-4862-bc25-c9b23a4e458f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:04.495536 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01afe5c5-c6e5-4862-bc25-c9b23a4e458f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:04.507143 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 250/502] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:04.508316 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d3226-f92d-4840-8c51-891fce74a6b5 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:04.508598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d3226-f92d-4840-8c51-891fce74a6b5 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:04.508873 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d3226-f92d-4840-8c51-891fce74a6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:04.527073 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-5f4d3226-f92d-4840-8c51-891fce74a6b5 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:04.528252 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-5f4d3226-f92d-4840-8c51-891fce74a6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:04.561317 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fda6fa-5fa7-4496-9d58-0b6cb8403d38 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:04.584607 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:04.593403 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 251/503] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:09:04.594257 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d3226-f92d-4840-8c51-891fce74a6b5 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:04.607578 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd3993b-c10d-4b7c-a440-4ab4a667a5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:04.608378 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd3993b-c10d-4b7c-a440-4ab4a667a5a7 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:04.608847 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd3993b-c10d-4b7c-a440-4ab4a667a5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:04.609200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd3993b-c10d-4b7c-a440-4ab4a667a5a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:04.609999 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd3993b-c10d-4b7c-a440-4ab4a667a5a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:04.610315 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd3993b-c10d-4b7c-a440-4ab4a667a5a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:04.612059 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd3993b-c10d-4b7c-a440-4ab4a667a5a7 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:04.624450 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 253/504] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:04.625245 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd3993b-c10d-4b7c-a440-4ab4a667a5a7 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:04.636989 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:04.693204 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d3226-f92d-4840-8c51-891fce74a6b5 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:04.723869 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd3993b-c10d-4b7c-a440-4ab4a667a5a7 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:04.726684 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542782a1-b3d8-4af6-b89d-82d28f54de88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:04.727323 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542782a1-b3d8-4af6-b89d-82d28f54de88 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:04.727607 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542782a1-b3d8-4af6-b89d-82d28f54de88 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:04.727880 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542782a1-b3d8-4af6-b89d-82d28f54de88 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:04.738508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542782a1-b3d8-4af6-b89d-82d28f54de88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:09:04.743366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-542782a1-b3d8-4af6-b89d-82d28f54de88 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:04.743366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:04.743366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:04.743366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:04.743366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:04.743366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:04.743366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:04.743366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:04.743366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:04.743366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:04.743366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:04.743366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:04.743366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:09:04.743366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:09:04.743366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:09:04.743366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:09:04.744351 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:09:04.744351 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:09:04.744351 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:09:04.744351 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:09:04.744351 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:04.744351 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:04.744351 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:04.744351 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:04.744351 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:04.744351 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:04.744351 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:04.744351 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:04.744351 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:04.744351 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:04.744351 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:04.744351 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:04.750172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:04.750172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:04.750172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:04.750172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:04.750172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:09:04.750172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:09:04.750172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:09:04.750172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:09:04.750172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:09:04.750172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:09:04.750172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:04.750172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:04.750172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 252/505] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 27 04:09:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:04.750172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542782a1-b3d8-4af6-b89d-82d28f54de88 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:04.758217 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7964c84-f096-482b-bcf9-421194cb9e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:04.758832 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7964c84-f096-482b-bcf9-421194cb9e4b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:04.759113 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7964c84-f096-482b-bcf9-421194cb9e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:04.759383 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7964c84-f096-482b-bcf9-421194cb9e4b None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:04.760101 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7964c84-f096-482b-bcf9-421194cb9e4b None admin] RBAC: Adding query filter params (name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:04.760401 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7964c84-f096-482b-bcf9-421194cb9e4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:04.761767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7964c84-f096-482b-bcf9-421194cb9e4b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:04.770770 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 254/506] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 27 04:09:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:04.771748 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7964c84-f096-482b-bcf9-421194cb9e4b None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:04.818713 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:04.843577 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542782a1-b3d8-4af6-b89d-82d28f54de88 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:04.845139 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:04.871535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7964c84-f096-482b-bcf9-421194cb9e4b None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:04.880574 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a51cd-afb4-4512-9c69-c71604188dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:04.881254 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a51cd-afb4-4512-9c69-c71604188dcd None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:04.881530 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a51cd-afb4-4512-9c69-c71604188dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:04.881801 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a51cd-afb4-4512-9c69-c71604188dcd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:04.888590 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a51cd-afb4-4512-9c69-c71604188dcd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:04.890127 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a51cd-afb4-4512-9c69-c71604188dcd None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:04.901729 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-697a51cd-afb4-4512-9c69-c71604188dcd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:04.901729 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:04.901729 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:04.901729 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:04.901729 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:04.901729 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:04.901729 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:04.901729 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:04.901729 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:04.901729 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:04.901729 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:04.901729 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:04.901729 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:09:04.901729 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:09:04.901729 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:09:04.902602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:09:04.902602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:04.902602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:04.902602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:04.902602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:04.902602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:04.902602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:04.902602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:04.902602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:04.902602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:04.902602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:04.902602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:04.902602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:04.902602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:04.902602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:04.902602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:04.902602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:04.903637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:09:04.903637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:09:04.903637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:09:04.903637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:09:04.903637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:09:04.903637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:09:04.903637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:04.903637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:04.903637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 253/507] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:04.903637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a51cd-afb4-4512-9c69-c71604188dcd None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:04.906804 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad11fd-1ec1-44eb-a500-b9ab03f98ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:04.907870 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad11fd-1ec1-44eb-a500-b9ab03f98ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:04.908157 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad11fd-1ec1-44eb-a500-b9ab03f98ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:04.908434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad11fd-1ec1-44eb-a500-b9ab03f98ed1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:04.919036 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad11fd-1ec1-44eb-a500-b9ab03f98ed1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:04.920488 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad11fd-1ec1-44eb-a500-b9ab03f98ed1 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:04.934973 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-01ad11fd-1ec1-44eb-a500-b9ab03f98ed1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:04.934973 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:04.934973 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:04.934973 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:04.934973 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:04.934973 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:04.934973 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:04.934973 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:04.934973 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:04.934973 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:04.934973 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:04.934973 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:04.934973 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:04.934973 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:04.934973 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:04.936029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:04.936029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:04.936029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:04.936029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:04.936029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:04.936029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:04.936029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:04.936029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:04.936029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:04.936029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:04.936029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:04.936029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:04.936029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:04.936029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:04.936029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:04.936029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:04.936029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:04.936873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:04.936873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:04.936873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:04.936873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:04.936873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 255/508] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:04.936873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad11fd-1ec1-44eb-a500-b9ab03f98ed1 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:05.003200 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a51cd-afb4-4512-9c69-c71604188dcd None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:05.029961 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:05.034461 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad11fd-1ec1-44eb-a500-b9ab03f98ed1 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:05.035984 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b8e93-2588-4987-ace3-748648aba6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:05.036627 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b8e93-2588-4987-ace3-748648aba6fe None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:05.036900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b8e93-2588-4987-ace3-748648aba6fe None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:05.037178 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b8e93-2588-4987-ace3-748648aba6fe None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:05.044517 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6b8e93-2588-4987-ace3-748648aba6fe None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:05.045956 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6b8e93-2588-4987-ace3-748648aba6fe None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:05.057480 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-1c6b8e93-2588-4987-ace3-748648aba6fe None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 04:09:05.057480 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:05.057480 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:05.057480 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:05.057480 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:05.057480 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:05.057480 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:05.057480 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:05.057480 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:05.057480 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:05.057480 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:05.057480 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:05.057480 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:09:05.057480 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:09:05.057480 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:09:05.057480 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:09:05.058419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:05.058419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:05.058419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:09:05.058419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:05.058419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:09:05.058419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:05.058419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:05.058419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:05.058419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:05.058419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:05.058419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:05.058419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:05.058419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:05.058419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:05.058419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:05.058419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:05.058419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:05.058419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:05.059474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:05.059474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:05.059474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:09:05.059474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:09:05.059474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:09:05.059474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:09:05.059474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:09:05.059474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:09:05.059474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 04:09:05.059474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:05.059474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 254/509] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:09:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:05.060569 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b8e93-2588-4987-ace3-748648aba6fe None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:05.072804 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a8e391-fabc-4071-991d-f03e9c9ef5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:05.073433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8e391-fabc-4071-991d-f03e9c9ef5d8 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:05.073715 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8e391-fabc-4071-991d-f03e9c9ef5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:05.074016 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8e391-fabc-4071-991d-f03e9c9ef5d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:05.074744 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a8e391-fabc-4071-991d-f03e9c9ef5d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:05.075053 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a8e391-fabc-4071-991d-f03e9c9ef5d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:05.076433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a8e391-fabc-4071-991d-f03e9c9ef5d8 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:05.084993 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 256/510] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 27 04:09:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:05.085585 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:05.085777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a8e391-fabc-4071-991d-f03e9c9ef5d8 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:05.161382 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b8e93-2588-4987-ace3-748648aba6fe None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:05.186117 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a8e391-fabc-4071-991d-f03e9c9ef5d8 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:05.196580 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc746742-3f83-4b70-b8dc-b2d6d2b63c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:05.197223 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc746742-3f83-4b70-b8dc-b2d6d2b63c27 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:05.197505 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc746742-3f83-4b70-b8dc-b2d6d2b63c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:05.197783 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc746742-3f83-4b70-b8dc-b2d6d2b63c27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:05.209492 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc746742-3f83-4b70-b8dc-b2d6d2b63c27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:05.210985 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc746742-3f83-4b70-b8dc-b2d6d2b63c27 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:05.219692 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e7d49-18be-45a7-9b16-296deaadf094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:05.220987 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e7d49-18be-45a7-9b16-296deaadf094 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:05.221276 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e7d49-18be-45a7-9b16-296deaadf094 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:05.221556 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e7d49-18be-45a7-9b16-296deaadf094 None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:05.222362 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e7d49-18be-45a7-9b16-296deaadf094 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:05.222681 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e7d49-18be-45a7-9b16-296deaadf094 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:05.224106 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e7d49-18be-45a7-9b16-296deaadf094 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:05.226780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-dc746742-3f83-4b70-b8dc-b2d6d2b63c27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:05.226780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:05.226780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:05.226780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:05.226780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:05.226780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:05.226780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:05.226780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:05.226780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:05.226780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:05.226780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:05.226780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:05.226780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:05.226780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:05.226780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:05.227570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:05.227570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:05.227570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:05.227570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:05.227570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:05.227570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:05.227570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:05.227570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:05.227570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:05.227570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:05.227570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:05.227570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:05.227570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:05.227570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:05.227570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:05.227570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:05.227570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:05.228407 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:05.228407 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:05.228407 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:05.228407 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:05.228407 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 255/511] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:05.229102 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc746742-3f83-4b70-b8dc-b2d6d2b63c27 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:05.245123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 257/512] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:09:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:05.246036 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e7d49-18be-45a7-9b16-296deaadf094 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:05.267385 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:05.306594 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:05.332078 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc746742-3f83-4b70-b8dc-b2d6d2b63c27 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:05.345673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e7d49-18be-45a7-9b16-296deaadf094 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:05.365209 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54efc2ac-98ba-4fcc-97a6-9169404dde52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:05.365824 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54efc2ac-98ba-4fcc-97a6-9169404dde52 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:05.366138 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54efc2ac-98ba-4fcc-97a6-9169404dde52 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:05.366410 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54efc2ac-98ba-4fcc-97a6-9169404dde52 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:05.367105 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54efc2ac-98ba-4fcc-97a6-9169404dde52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:05.367406 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54efc2ac-98ba-4fcc-97a6-9169404dde52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:05.368761 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54efc2ac-98ba-4fcc-97a6-9169404dde52 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:05.380710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 256/513] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:05.380845 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c736f26-9e00-4ca7-b6b0-6f51f81c8e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:05.381453 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54efc2ac-98ba-4fcc-97a6-9169404dde52 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:05.381870 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c736f26-9e00-4ca7-b6b0-6f51f81c8e8d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:05.382202 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c736f26-9e00-4ca7-b6b0-6f51f81c8e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:05.382486 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c736f26-9e00-4ca7-b6b0-6f51f81c8e8d None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:05.383309 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c736f26-9e00-4ca7-b6b0-6f51f81c8e8d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:05.383624 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c736f26-9e00-4ca7-b6b0-6f51f81c8e8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:05.386458 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c736f26-9e00-4ca7-b6b0-6f51f81c8e8d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:05.427832 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 258/514] 10.222.0.46 () {56 vars in 1137 bytes} [Wed Jul 27 04:09:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:05.428919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c736f26-9e00-4ca7-b6b0-6f51f81c8e8d None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:05.486745 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54efc2ac-98ba-4fcc-97a6-9169404dde52 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:05.510618 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:05.520729 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef836d9f-d639-48b3-b4ab-fb4935ab3ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:05.521317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef836d9f-d639-48b3-b4ab-fb4935ab3ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:05.521601 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef836d9f-d639-48b3-b4ab-fb4935ab3ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:05.521877 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef836d9f-d639-48b3-b4ab-fb4935ab3ec5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:05.523795 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:05.532277 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef836d9f-d639-48b3-b4ab-fb4935ab3ec5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:05.533247 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c736f26-9e00-4ca7-b6b0-6f51f81c8e8d None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:05.533906 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef836d9f-d639-48b3-b4ab-fb4935ab3ec5 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:05.549180 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-ef836d9f-d639-48b3-b4ab-fb4935ab3ec5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:05.549180 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:05.549180 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:05.549180 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:05.549180 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:05.549180 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:05.549180 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:05.549180 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:05.549180 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:05.549180 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:05.549180 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:05.549180 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:05.549180 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:05.549180 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:05.549180 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:05.550084 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:05.550084 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:05.550084 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:05.550084 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:05.550084 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:05.550084 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:05.550084 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:05.550084 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:05.550084 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:05.550084 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:05.550084 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:05.550084 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:05.550084 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:05.550084 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:05.550084 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:05.550084 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:05.550084 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:05.550928 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:05.550928 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:05.550928 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:05.550928 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:05.550928 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 257/515] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:05.551198 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef836d9f-d639-48b3-b4ab-fb4935ab3ec5 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:05.573537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb92c33-27fb-4c77-913a-fbc192ce5ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:05.574198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb92c33-27fb-4c77-913a-fbc192ce5ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:05.574483 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb92c33-27fb-4c77-913a-fbc192ce5ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:05.574761 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb92c33-27fb-4c77-913a-fbc192ce5ab5 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:05.575469 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb92c33-27fb-4c77-913a-fbc192ce5ab5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:05.575777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb92c33-27fb-4c77-913a-fbc192ce5ab5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:05.577165 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb92c33-27fb-4c77-913a-fbc192ce5ab5 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:05.588141 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 259/516] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:05.588902 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb92c33-27fb-4c77-913a-fbc192ce5ab5 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:05.654423 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef836d9f-d639-48b3-b4ab-fb4935ab3ec5 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:05.687722 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b811fa8-5b8e-4130-845e-d0ea19f6b146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:05.687946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb92c33-27fb-4c77-913a-fbc192ce5ab5 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:05.688694 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b811fa8-5b8e-4130-845e-d0ea19f6b146 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:05.689134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b811fa8-5b8e-4130-845e-d0ea19f6b146 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:05.689621 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b811fa8-5b8e-4130-845e-d0ea19f6b146 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:05.691060 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b811fa8-5b8e-4130-845e-d0ea19f6b146 None admin] RBAC: Adding query filter params (role.id=6521132834a64b029695bbdd5eb251d2, scope.project.id=e646d320aa5a4f78bfccaf941cf46478, user.id=9dc9ea8f89c14a2a8e3272f750647dfd) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:05.691373 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b811fa8-5b8e-4130-845e-d0ea19f6b146 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:05.692830 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b811fa8-5b8e-4130-845e-d0ea19f6b146 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:05.698262 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 258/517] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:09:05 2022] GET /identity/v3/role_assignments?user.id=9dc9ea8f89c14a2a8e3272f750647dfd&scope.project.id=e646d320aa5a4f78bfccaf941cf46478&role.id=6521132834a64b029695bbdd5eb251d2 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:05.699050 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b811fa8-5b8e-4130-845e-d0ea19f6b146 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:05.739880 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c95361-e014-4170-a828-7f7c676b6da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:05.740574 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c95361-e014-4170-a828-7f7c676b6da5 None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:05.740862 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c95361-e014-4170-a828-7f7c676b6da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:05.742009 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c95361-e014-4170-a828-7f7c676b6da5 None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:05.742009 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c95361-e014-4170-a828-7f7c676b6da5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:05.743405 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c95361-e014-4170-a828-7f7c676b6da5 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:05.755688 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:05.759621 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:05.770140 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 260/518] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 27 04:09:05 2022] POST /identity/v3/users => generated 247 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:09:05.783303 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c95361-e014-4170-a828-7f7c676b6da5 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:05.836671 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b811fa8-5b8e-4130-845e-d0ea19f6b146 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:05.894548 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f306e549-365c-4234-9dd3-550c50e8d8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:05.895203 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306e549-365c-4234-9dd3-550c50e8d8ef None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:05.895486 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306e549-365c-4234-9dd3-550c50e8d8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:05.895760 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306e549-365c-4234-9dd3-550c50e8d8ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:05.935950 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c95361-e014-4170-a828-7f7c676b6da5 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:05.974790 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a62e8-37af-4533-a3b4-3911a9f11825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:05.975424 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a62e8-37af-4533-a3b4-3911a9f11825 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:05.975533 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a62e8-37af-4533-a3b4-3911a9f11825 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:05.975806 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a62e8-37af-4533-a3b4-3911a9f11825 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:05.977025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a62e8-37af-4533-a3b4-3911a9f11825 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:05.977025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a62e8-37af-4533-a3b4-3911a9f11825 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:05.978237 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a62e8-37af-4533-a3b4-3911a9f11825 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:05.989112 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f306e549-365c-4234-9dd3-550c50e8d8ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:05.991745 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f306e549-365c-4234-9dd3-550c50e8d8ef None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:05.991862 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 261/519] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:05.991862 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a62e8-37af-4533-a3b4-3911a9f11825 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:06.046376 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:06.065885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:06.102743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 259/520] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:09:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:09:06.108242 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f306e549-365c-4234-9dd3-550c50e8d8ef None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:06.123408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a62e8-37af-4533-a3b4-3911a9f11825 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:06.167032 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00e6335-f3c3-4b87-b471-d7360f09cb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:06.167786 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e6335-f3c3-4b87-b471-d7360f09cb5b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:06.168228 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e6335-f3c3-4b87-b471-d7360f09cb5b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:06.168571 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e6335-f3c3-4b87-b471-d7360f09cb5b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:06.178353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00e6335-f3c3-4b87-b471-d7360f09cb5b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:06.181004 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00e6335-f3c3-4b87-b471-d7360f09cb5b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:06.200007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-e00e6335-f3c3-4b87-b471-d7360f09cb5b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 04:09:06.200007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:06.200007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:06.200007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:06.200007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:06.200007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:06.200007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:06.200007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:06.200007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:06.200007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:06.200007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:06.200007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:06.200007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:09:06.200007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:09:06.200007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:09:06.200007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:09:06.200925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:06.200925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:06.200925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:09:06.200925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:06.200925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:09:06.200925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:06.200925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:06.200925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:06.200925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:06.200925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:06.200925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:06.200925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:06.200925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:06.200925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:06.200925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:06.200925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:06.200925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:06.200925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:06.201788 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:06.201788 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:06.201788 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:09:06.201788 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:09:06.201788 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:09:06.201788 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:09:06.201788 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:09:06.201788 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:09:06.201788 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 04:09:06.201788 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:06.201788 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 262/521] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 27 04:09:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:06.202491 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00e6335-f3c3-4b87-b471-d7360f09cb5b None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:06.238097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f306e549-365c-4234-9dd3-550c50e8d8ef None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:06.286004 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6ceb8-2bfb-4d1e-9d06-5f81dee02e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:06.287015 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6ceb8-2bfb-4d1e-9d06-5f81dee02e2e None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:06.287302 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6ceb8-2bfb-4d1e-9d06-5f81dee02e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:06.287733 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6ceb8-2bfb-4d1e-9d06-5f81dee02e2e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:06.300343 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6ceb8-2bfb-4d1e-9d06-5f81dee02e2e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:06.301889 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6ceb8-2bfb-4d1e-9d06-5f81dee02e2e None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:06.312347 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00e6335-f3c3-4b87-b471-d7360f09cb5b None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:06.315177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-5ee6ceb8-2bfb-4d1e-9d06-5f81dee02e2e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 27 04:09:06.315177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:06.315177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:06.315177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:06.315177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:06.315177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:06.315177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:06.315177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:06.315177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:06.315177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:06.315177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:06.315177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:06.315177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:09:06.315177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:09:06.315177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:09:06.315177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:09:06.316909 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:06.316909 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:06.316909 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:09:06.316909 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:06.316909 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:09:06.316909 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:06.316909 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:06.316909 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:06.316909 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:06.316909 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:06.316909 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:06.316909 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:06.316909 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:06.316909 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:06.316909 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:06.316909 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:06.316909 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:06.316909 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:06.318862 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:06.318862 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:06.318862 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:09:06.318862 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:09:06.318862 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:09:06.318862 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:09:06.318862 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:09:06.318862 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:09:06.318862 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 27 04:09:06.318862 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:06.318862 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 260/522] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:09:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:06.320307 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6ceb8-2bfb-4d1e-9d06-5f81dee02e2e None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:06.332722 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:06.346533 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:06.365663 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443bc8db-541b-4a69-a8d5-9ec92baa1e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:06.366403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bc8db-541b-4a69-a8d5-9ec92baa1e8b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:06.367130 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bc8db-541b-4a69-a8d5-9ec92baa1e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:06.367423 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bc8db-541b-4a69-a8d5-9ec92baa1e8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:06.384682 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443bc8db-541b-4a69-a8d5-9ec92baa1e8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:06.387031 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443bc8db-541b-4a69-a8d5-9ec92baa1e8b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:06.410210 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-443bc8db-541b-4a69-a8d5-9ec92baa1e8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:06.410210 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:06.410210 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:06.410210 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:06.410210 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:06.410210 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:06.410210 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:06.410210 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:06.410210 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:06.410210 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:06.410210 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:06.410210 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:06.410210 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:06.410210 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:06.410210 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:06.411638 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:06.411638 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:06.411638 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:06.411638 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:06.411638 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:06.411638 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:06.411638 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:06.411638 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:06.411638 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:06.411638 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:06.411638 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:06.411638 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:06.411638 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:06.411638 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:06.411638 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:06.411638 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:06.411638 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:06.413279 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:06.413279 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:06.413279 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:06.413279 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:06.413279 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 263/523] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:06.413279 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443bc8db-541b-4a69-a8d5-9ec92baa1e8b None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:06.439530 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6ceb8-2bfb-4d1e-9d06-5f81dee02e2e None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:06.480422 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39218ca7-42bb-4538-913f-370be2930962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:06.481198 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39218ca7-42bb-4538-913f-370be2930962 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:06.481554 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39218ca7-42bb-4538-913f-370be2930962 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:06.481925 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39218ca7-42bb-4538-913f-370be2930962 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:06.482848 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39218ca7-42bb-4538-913f-370be2930962 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:06.483228 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39218ca7-42bb-4538-913f-370be2930962 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:06.484791 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39218ca7-42bb-4538-913f-370be2930962 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:06.504212 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 261/524] 10.222.0.46 () {56 vars in 1139 bytes} [Wed Jul 27 04:09:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:06.505232 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39218ca7-42bb-4538-913f-370be2930962 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:06.545603 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443bc8db-541b-4a69-a8d5-9ec92baa1e8b None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:06.585382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f83156-872b-499c-b1fb-c71e06167f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:06.586380 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f83156-872b-499c-b1fb-c71e06167f06 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:06.586760 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f83156-872b-499c-b1fb-c71e06167f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:06.587108 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f83156-872b-499c-b1fb-c71e06167f06 None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:06.587991 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f83156-872b-499c-b1fb-c71e06167f06 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:06.588377 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f83156-872b-499c-b1fb-c71e06167f06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:06.590006 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f83156-872b-499c-b1fb-c71e06167f06 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:06.590480 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:06.612444 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 264/525] 10.222.0.46 () {56 vars in 1137 bytes} [Wed Jul 27 04:09:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:06.613410 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f83156-872b-499c-b1fb-c71e06167f06 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:06.631835 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39218ca7-42bb-4538-913f-370be2930962 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:06.639931 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:06.671485 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e166ec4-ad9c-4775-9a91-95aa40fc23ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:06.672386 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e166ec4-ad9c-4775-9a91-95aa40fc23ce None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:06.672810 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e166ec4-ad9c-4775-9a91-95aa40fc23ce None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:06.673303 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e166ec4-ad9c-4775-9a91-95aa40fc23ce None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:06.674554 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e166ec4-ad9c-4775-9a91-95aa40fc23ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:06.675096 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e166ec4-ad9c-4775-9a91-95aa40fc23ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:06.676633 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e166ec4-ad9c-4775-9a91-95aa40fc23ce None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:06.689672 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 262/526] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:06.690754 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e166ec4-ad9c-4775-9a91-95aa40fc23ce None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:06.727815 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f83156-872b-499c-b1fb-c71e06167f06 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:06.764455 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70745019-9c6a-4e08-8450-c6c53b58f895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:06.765222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70745019-9c6a-4e08-8450-c6c53b58f895 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:06.765514 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70745019-9c6a-4e08-8450-c6c53b58f895 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:06.765794 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70745019-9c6a-4e08-8450-c6c53b58f895 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:06.801305 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e166ec4-ad9c-4775-9a91-95aa40fc23ce None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:06.829768 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:06.839529 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83677cba-e138-488b-b2f4-20d8537683b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:06.840519 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83677cba-e138-488b-b2f4-20d8537683b5 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:06.841067 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83677cba-e138-488b-b2f4-20d8537683b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:06.841484 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83677cba-e138-488b-b2f4-20d8537683b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:06.843787 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70745019-9c6a-4e08-8450-c6c53b58f895 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:06.846078 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70745019-9c6a-4e08-8450-c6c53b58f895 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:06.872323 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:06.919815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83677cba-e138-488b-b2f4-20d8537683b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:06.921652 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83677cba-e138-488b-b2f4-20d8537683b5 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:06.962611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 265/527] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:09:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:09:06.963549 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70745019-9c6a-4e08-8450-c6c53b58f895 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:07.034788 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 263/528] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:09:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:09:07.035937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83677cba-e138-488b-b2f4-20d8537683b5 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:07.066811 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:07.102655 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70745019-9c6a-4e08-8450-c6c53b58f895 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:07.134385 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:07.150270 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc0d12-dc24-467b-9184-30540e0a9818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:07.150933 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc0d12-dc24-467b-9184-30540e0a9818 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:07.151222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc0d12-dc24-467b-9184-30540e0a9818 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:07.151506 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc0d12-dc24-467b-9184-30540e0a9818 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:07.152461 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83677cba-e138-488b-b2f4-20d8537683b5 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:07.159668 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc0d12-dc24-467b-9184-30540e0a9818 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:07.161146 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc0d12-dc24-467b-9184-30540e0a9818 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:07.178741 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-83bc0d12-dc24-467b-9184-30540e0a9818 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:07.178741 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:07.178741 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:07.178741 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:07.178741 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:07.178741 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:07.178741 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:07.178741 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:07.178741 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:07.178741 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:07.178741 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:07.178741 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:07.178741 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:09:07.178741 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:09:07.178741 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:09:07.179699 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:09:07.179699 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:07.179699 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:07.179699 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:07.179699 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:07.179699 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:07.179699 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:07.179699 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:07.179699 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:07.179699 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:07.179699 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:07.179699 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:07.179699 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:07.179699 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:07.179699 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:07.179699 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:07.179699 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:07.180610 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:09:07.180610 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:09:07.180610 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:09:07.180610 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:09:07.180610 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:09:07.180610 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:09:07.180610 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:07.180610 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:07.182047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 266/529] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:07.183001 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc0d12-dc24-467b-9184-30540e0a9818 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:07.200895 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647fc49e-2f01-4c45-877d-68d4f0421283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:07.201869 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fc49e-2f01-4c45-877d-68d4f0421283 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:07.202458 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fc49e-2f01-4c45-877d-68d4f0421283 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:07.202982 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fc49e-2f01-4c45-877d-68d4f0421283 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:07.215276 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647fc49e-2f01-4c45-877d-68d4f0421283 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:07.217062 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647fc49e-2f01-4c45-877d-68d4f0421283 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:07.233398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-647fc49e-2f01-4c45-877d-68d4f0421283 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:07.233398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:07.233398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:07.233398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:07.233398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:07.233398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:07.233398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:07.233398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:07.233398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:07.233398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:07.233398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:07.233398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:07.233398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:07.233398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:07.233398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:07.234327 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:07.234327 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:07.234327 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:07.234327 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:07.234327 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:07.234327 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:07.234327 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:07.234327 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:07.234327 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:07.234327 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:07.234327 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:07.234327 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:07.234327 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:07.234327 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:07.234327 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:07.234327 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:07.234327 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:07.235164 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:07.235164 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:07.235164 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:07.235164 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:07.242965 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 264/530] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:07.244113 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647fc49e-2f01-4c45-877d-68d4f0421283 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:07.312338 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc0d12-dc24-467b-9184-30540e0a9818 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:07.341317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:07.355121 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd3023-5a5e-4fd4-8d68-91b5f6c0e966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:07.356030 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd3023-5a5e-4fd4-8d68-91b5f6c0e966 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:07.356492 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd3023-5a5e-4fd4-8d68-91b5f6c0e966 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:07.356950 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd3023-5a5e-4fd4-8d68-91b5f6c0e966 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:07.367693 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647fc49e-2f01-4c45-877d-68d4f0421283 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:07.369793 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd3023-5a5e-4fd4-8d68-91b5f6c0e966 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:07.371559 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd3023-5a5e-4fd4-8d68-91b5f6c0e966 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:07.388116 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-23cd3023-5a5e-4fd4-8d68-91b5f6c0e966 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:07.388116 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:07.388116 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:07.388116 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:07.388116 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:07.388116 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:07.388116 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:07.388116 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:07.388116 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:07.388116 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:07.388116 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:07.388116 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:07.388116 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:07.388116 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:07.388116 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:07.389408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:07.389408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:07.389408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:07.389408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:07.389408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:07.389408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:07.389408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:07.389408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:07.389408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:07.389408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:07.389408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:07.389408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:07.389408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:07.389408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:07.389408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:07.389408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:07.389408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:07.390264 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:07.390264 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:07.390264 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:07.390264 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:07.393478 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 267/531] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:07.394012 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd3023-5a5e-4fd4-8d68-91b5f6c0e966 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:07.409915 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535718cc-7add-4f78-8a55-56dcf04fe5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:07.410151 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535718cc-7add-4f78-8a55-56dcf04fe5cc None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:07.410384 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535718cc-7add-4f78-8a55-56dcf04fe5cc None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:07.410547 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535718cc-7add-4f78-8a55-56dcf04fe5cc None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:07.411590 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535718cc-7add-4f78-8a55-56dcf04fe5cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:07.411919 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535718cc-7add-4f78-8a55-56dcf04fe5cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:07.413372 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535718cc-7add-4f78-8a55-56dcf04fe5cc None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:07.434907 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:07.443143 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 265/532] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:09:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:07.445909 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535718cc-7add-4f78-8a55-56dcf04fe5cc None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:07.522009 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd3023-5a5e-4fd4-8d68-91b5f6c0e966 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:07.566000 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a725ed12-4f51-49b4-8306-8da880611902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:07.566000 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725ed12-4f51-49b4-8306-8da880611902 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:07.566000 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725ed12-4f51-49b4-8306-8da880611902 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:07.566000 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725ed12-4f51-49b4-8306-8da880611902 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:07.566362 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a725ed12-4f51-49b4-8306-8da880611902 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:07.566362 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a725ed12-4f51-49b4-8306-8da880611902 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:07.569995 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a725ed12-4f51-49b4-8306-8da880611902 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:07.577997 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535718cc-7add-4f78-8a55-56dcf04fe5cc None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:07.586001 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 268/533] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:07.586001 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a725ed12-4f51-49b4-8306-8da880611902 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:07.617633 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58808301-6b68-4d57-a166-ca31ee10f6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:07.618418 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58808301-6b68-4d57-a166-ca31ee10f6ec None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:07.618761 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58808301-6b68-4d57-a166-ca31ee10f6ec None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:07.619099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58808301-6b68-4d57-a166-ca31ee10f6ec None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:07.619871 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58808301-6b68-4d57-a166-ca31ee10f6ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:07.620327 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58808301-6b68-4d57-a166-ca31ee10f6ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:07.622007 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58808301-6b68-4d57-a166-ca31ee10f6ec None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:07.631710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:07.636343 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 266/534] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:07.637366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58808301-6b68-4d57-a166-ca31ee10f6ec None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:07.703171 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:07.710950 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a725ed12-4f51-49b4-8306-8da880611902 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:07.756842 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f133a9-6b99-46a7-963e-3d950a9022b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:07.757254 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58808301-6b68-4d57-a166-ca31ee10f6ec None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:07.757883 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f133a9-6b99-46a7-963e-3d950a9022b0 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:07.758337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f133a9-6b99-46a7-963e-3d950a9022b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:07.759042 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f133a9-6b99-46a7-963e-3d950a9022b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:07.760914 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f133a9-6b99-46a7-963e-3d950a9022b0 None admin] RBAC: Adding query filter params (role.id=6521132834a64b029695bbdd5eb251d2, scope.project.id=e646d320aa5a4f78bfccaf941cf46478, user.id=134ef452d7814c73b7c371a1cdbcbc00) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:07.761235 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f133a9-6b99-46a7-963e-3d950a9022b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:07.762714 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f133a9-6b99-46a7-963e-3d950a9022b0 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:07.775711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 269/535] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:09:07 2022] GET /identity/v3/role_assignments?user.id=134ef452d7814c73b7c371a1cdbcbc00&scope.project.id=e646d320aa5a4f78bfccaf941cf46478&role.id=6521132834a64b029695bbdd5eb251d2 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:07.775711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f133a9-6b99-46a7-963e-3d950a9022b0 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:07.795766 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28297f6-2ad8-438a-a6fe-864f2a11bea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:07.796436 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28297f6-2ad8-438a-a6fe-864f2a11bea2 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:07.796855 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28297f6-2ad8-438a-a6fe-864f2a11bea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:07.797042 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28297f6-2ad8-438a-a6fe-864f2a11bea2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:07.805410 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28297f6-2ad8-438a-a6fe-864f2a11bea2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:07.807045 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28297f6-2ad8-438a-a6fe-864f2a11bea2 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:07.820213 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-d28297f6-2ad8-438a-a6fe-864f2a11bea2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:07.820213 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:07.820213 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:07.820213 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:07.820213 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:07.820213 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:07.820213 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:07.820213 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:07.820213 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:07.820213 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:07.820213 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:07.820213 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:07.820213 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:09:07.820213 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:09:07.820213 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:09:07.821130 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:09:07.821130 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:07.821130 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:07.821130 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:07.821130 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:07.821130 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:07.821130 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:07.821130 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:07.821130 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:07.821130 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:07.821130 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:07.821130 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:07.821130 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:07.821130 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:07.821130 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:07.821130 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:07.821130 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:07.821966 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:09:07.821966 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:09:07.821966 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:09:07.821966 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:09:07.821966 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:09:07.821966 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:09:07.821966 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:07.821966 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:07.821966 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 267/536] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:07.822917 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6cf93e-561a-47d7-98c7-70425cb77edd None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:07.823011 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6cf93e-561a-47d7-98c7-70425cb77edd None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:07.823616 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6cf93e-561a-47d7-98c7-70425cb77edd None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:07.828601 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 268/537] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:07.830496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa54b38-ec76-4c08-bbc6-46c024452e15 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:07.830955 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa54b38-ec76-4c08-bbc6-46c024452e15 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:07.831477 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa54b38-ec76-4c08-bbc6-46c024452e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:07.850892 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-faa54b38-ec76-4c08-bbc6-46c024452e15 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:07.852123 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-faa54b38-ec76-4c08-bbc6-46c024452e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:07.895350 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f133a9-6b99-46a7-963e-3d950a9022b0 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:07.922445 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:07.936689 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 269/538] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:09:07.937598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa54b38-ec76-4c08-bbc6-46c024452e15 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:07.941999 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d112a8-1347-4f50-8656-e6964d5596cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:07.942980 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d112a8-1347-4f50-8656-e6964d5596cb None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:07.943488 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d112a8-1347-4f50-8656-e6964d5596cb None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:07.943984 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d112a8-1347-4f50-8656-e6964d5596cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:07.951691 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d112a8-1347-4f50-8656-e6964d5596cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:07.953525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d112a8-1347-4f50-8656-e6964d5596cb None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:07.966969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-48d112a8-1347-4f50-8656-e6964d5596cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:07.966969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:07.966969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:07.966969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:07.966969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:07.966969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:07.966969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:07.966969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:07.966969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:07.966969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:07.966969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:07.966969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:07.966969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:09:07.966969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:09:07.966969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:09:07.967869 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:09:07.967869 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:07.967869 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:07.967869 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:07.967869 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:07.967869 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:07.967869 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:07.967869 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:07.967869 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:07.967869 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:07.967869 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:07.967869 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:07.967869 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:07.967869 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:07.967869 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:07.967869 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:07.967869 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:07.968795 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:09:07.968795 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:09:07.968795 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:09:07.968795 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:09:07.968795 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:09:07.968795 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:09:07.968795 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:07.968795 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:07.971748 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 270/539] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:07.973282 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350535a0-828e-4816-b747-7940f6df0730 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:07.973641 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350535a0-828e-4816-b747-7940f6df0730 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:07.974012 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350535a0-828e-4816-b747-7940f6df0730 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:07.975616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 271/540] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:07.976404 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350535a0-828e-4816-b747-7940f6df0730 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:08.000679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:08.051831 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa54b38-ec76-4c08-bbc6-46c024452e15 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:08.089745 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eddb51-3738-4dcd-82d9-dc7dc8a253b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:08.090520 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eddb51-3738-4dcd-82d9-dc7dc8a253b1 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:08.090818 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eddb51-3738-4dcd-82d9-dc7dc8a253b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:08.091085 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eddb51-3738-4dcd-82d9-dc7dc8a253b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:08.091661 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350535a0-828e-4816-b747-7940f6df0730 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:08.091851 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eddb51-3738-4dcd-82d9-dc7dc8a253b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:08.092159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eddb51-3738-4dcd-82d9-dc7dc8a253b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:08.093601 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eddb51-3738-4dcd-82d9-dc7dc8a253b1 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:08.107047 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 270/541] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:09:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:08.107883 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eddb51-3738-4dcd-82d9-dc7dc8a253b1 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:08.133930 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d4375b-0fcc-4c97-94c5-7378fadcf648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:08.134603 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4375b-0fcc-4c97-94c5-7378fadcf648 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:08.134882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4375b-0fcc-4c97-94c5-7378fadcf648 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:08.135143 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4375b-0fcc-4c97-94c5-7378fadcf648 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:08.147257 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d4375b-0fcc-4c97-94c5-7378fadcf648 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:09:08.152700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-c5d4375b-0fcc-4c97-94c5-7378fadcf648 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:08.152700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:08.152700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:08.152700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:08.152700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:08.152700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:08.152700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:08.152700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:08.152700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:08.152700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:08.152700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:08.152700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:08.152700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:09:08.152700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:09:08.152700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:09:08.152700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:09:08.153631 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:09:08.153631 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:09:08.153631 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:09:08.153631 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:09:08.153631 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:08.153631 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:08.153631 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:08.153631 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:08.153631 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:08.153631 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:08.153631 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:08.153631 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:08.153631 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:08.153631 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:08.153631 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:08.153631 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:08.154819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:08.154819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:08.154819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:08.154819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:08.154819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:09:08.154819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:09:08.154819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:09:08.154819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:09:08.154819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:09:08.154819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:09:08.154819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:08.154819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:08.154819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 272/542] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 27 04:09:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:08.155467 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851da8b-9057-464d-b328-12e7d249e009 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:08.155467 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851da8b-9057-464d-b328-12e7d249e009 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:08.155850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851da8b-9057-464d-b328-12e7d249e009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:08.165371 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:08.176436 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-3851da8b-9057-464d-b328-12e7d249e009 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:08.177643 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-3851da8b-9057-464d-b328-12e7d249e009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:08.222920 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eddb51-3738-4dcd-82d9-dc7dc8a253b1 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:08.250681 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 273/543] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:09:08.252384 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3851da8b-9057-464d-b328-12e7d249e009 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:08.265610 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b170e0-23e7-4a8e-b031-3965f7cb8c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:08.266602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b170e0-23e7-4a8e-b031-3965f7cb8c59 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:08.267852 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:08.269067 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b170e0-23e7-4a8e-b031-3965f7cb8c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:08.269651 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b170e0-23e7-4a8e-b031-3965f7cb8c59 None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:08.270938 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b170e0-23e7-4a8e-b031-3965f7cb8c59 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:08.271169 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b170e0-23e7-4a8e-b031-3965f7cb8c59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:08.272849 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b170e0-23e7-4a8e-b031-3965f7cb8c59 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:08.288474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 271/544] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:09:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:08.289635 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b170e0-23e7-4a8e-b031-3965f7cb8c59 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:08.362491 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3851da8b-9057-464d-b328-12e7d249e009 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:08.399790 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b170e0-23e7-4a8e-b031-3965f7cb8c59 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:08.400426 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a813fc-2368-47d5-9257-480b42443895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:08.401051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a813fc-2368-47d5-9257-480b42443895 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:08.401334 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a813fc-2368-47d5-9257-480b42443895 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:08.401609 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a813fc-2368-47d5-9257-480b42443895 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:08.402444 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a813fc-2368-47d5-9257-480b42443895 None admin] RBAC: Adding query filter params (role.id=6521132834a64b029695bbdd5eb251d2, scope.project.id=e646d320aa5a4f78bfccaf941cf46478, user.id=b4847f3736d844d78a206b7eab5f5fcd) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:08.402742 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a813fc-2368-47d5-9257-480b42443895 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:08.404160 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a813fc-2368-47d5-9257-480b42443895 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:08.410546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 274/545] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:09:08 2022] GET /identity/v3/role_assignments?user.id=b4847f3736d844d78a206b7eab5f5fcd&scope.project.id=e646d320aa5a4f78bfccaf941cf46478&role.id=6521132834a64b029695bbdd5eb251d2 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:08.411329 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a813fc-2368-47d5-9257-480b42443895 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:08.423052 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:08.442814 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99188a-10fc-4a98-aa6c-a4feda4f2d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:08.443455 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99188a-10fc-4a98-aa6c-a4feda4f2d2d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:08.443986 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99188a-10fc-4a98-aa6c-a4feda4f2d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:08.443986 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99188a-10fc-4a98-aa6c-a4feda4f2d2d None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:08.444723 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99188a-10fc-4a98-aa6c-a4feda4f2d2d None admin] RBAC: Adding query filter params (name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:08.445030 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99188a-10fc-4a98-aa6c-a4feda4f2d2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:08.446499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99188a-10fc-4a98-aa6c-a4feda4f2d2d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:08.458042 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 272/546] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 27 04:09:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:08.458042 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99188a-10fc-4a98-aa6c-a4feda4f2d2d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:08.529288 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:08.567951 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a813fc-2368-47d5-9257-480b42443895 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:08.576910 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99188a-10fc-4a98-aa6c-a4feda4f2d2d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:08.614442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c29a94-956d-4f4c-949e-fb76171a1900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:08.615094 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c29a94-956d-4f4c-949e-fb76171a1900 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:08.615376 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c29a94-956d-4f4c-949e-fb76171a1900 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:08.615652 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c29a94-956d-4f4c-949e-fb76171a1900 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:08.618368 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a561cc3-1263-4baa-8e56-4b350cd2080d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:08.618986 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a561cc3-1263-4baa-8e56-4b350cd2080d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:08.619262 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a561cc3-1263-4baa-8e56-4b350cd2080d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:08.619533 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a561cc3-1263-4baa-8e56-4b350cd2080d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:08.620335 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a561cc3-1263-4baa-8e56-4b350cd2080d None admin] RBAC: Adding query filter params (role.id=6521132834a64b029695bbdd5eb251d2, scope.project.id=e646d320aa5a4f78bfccaf941cf46478, user.id=3e98a41a0f304fc2bb36a18a77a2d729) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:08.620641 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a561cc3-1263-4baa-8e56-4b350cd2080d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:08.622102 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a561cc3-1263-4baa-8e56-4b350cd2080d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:08.627379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c29a94-956d-4f4c-949e-fb76171a1900 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:09:08.631941 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 273/547] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:09:08 2022] GET /identity/v3/role_assignments?user.id=3e98a41a0f304fc2bb36a18a77a2d729&scope.project.id=e646d320aa5a4f78bfccaf941cf46478&role.id=6521132834a64b029695bbdd5eb251d2 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:08.632917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-43c29a94-956d-4f4c-949e-fb76171a1900 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:08.632917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:08.632917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:08.632917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:08.632917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:08.632917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:08.632917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:08.632917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:08.632917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:08.632917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:08.632917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:08.632917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:08.632917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:09:08.632917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:09:08.632917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:09:08.632917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:09:08.633800 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:09:08.633800 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:09:08.633800 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:09:08.633800 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:09:08.633800 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:08.633800 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:08.633800 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:08.633800 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:08.633800 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:08.633800 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:08.633800 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:08.633800 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:08.633800 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:08.633800 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:08.633800 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:08.633800 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:08.634611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:08.634611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:08.634611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:08.634611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:08.634611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:09:08.634611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:09:08.634611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:09:08.634611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:09:08.634611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:09:08.634611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:09:08.634611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:08.634611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:08.634611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 275/548] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 27 04:09:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:08.635868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a561cc3-1263-4baa-8e56-4b350cd2080d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:08.642490 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c29a94-956d-4f4c-949e-fb76171a1900 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:08.712062 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:08.817722 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a561cc3-1263-4baa-8e56-4b350cd2080d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:08.864182 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c29a94-956d-4f4c-949e-fb76171a1900 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:08.885568 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796ef45-6d5c-4e06-b545-5de73b87bdc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:08.886618 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796ef45-6d5c-4e06-b545-5de73b87bdc7 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:08.887097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796ef45-6d5c-4e06-b545-5de73b87bdc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:08.887658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796ef45-6d5c-4e06-b545-5de73b87bdc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:08.907112 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7796ef45-6d5c-4e06-b545-5de73b87bdc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:08.909728 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7796ef45-6d5c-4e06-b545-5de73b87bdc7 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:08.935864 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-7796ef45-6d5c-4e06-b545-5de73b87bdc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:08.935864 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:08.935864 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:08.935864 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:08.935864 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:08.935864 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:08.935864 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:08.935864 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:08.935864 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:08.935864 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:08.935864 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:08.935864 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:08.935864 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:08.935864 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:08.935864 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:08.936783 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:08.936783 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:08.936783 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:08.936783 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:08.936783 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:08.936783 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:08.936783 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:08.936783 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:08.936783 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:08.936783 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:08.936783 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:08.936783 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:08.936783 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:08.936783 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:08.936783 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:08.936783 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:08.936783 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:08.939146 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:08.939146 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:08.939146 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:08.939146 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:08.939146 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 274/549] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:08.939533 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83c9bd-47fd-4252-8b1a-14b22c850ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:08.939705 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83c9bd-47fd-4252-8b1a-14b22c850ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:08.940679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83c9bd-47fd-4252-8b1a-14b22c850ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:08.940891 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83c9bd-47fd-4252-8b1a-14b22c850ca8 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:08.942089 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a83c9bd-47fd-4252-8b1a-14b22c850ca8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:08.942089 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a83c9bd-47fd-4252-8b1a-14b22c850ca8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:08.943660 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a83c9bd-47fd-4252-8b1a-14b22c850ca8 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:08.945297 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:08.947479 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796ef45-6d5c-4e06-b545-5de73b87bdc7 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:08.955591 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 276/550] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 27 04:09:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:08.966188 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83c9bd-47fd-4252-8b1a-14b22c850ca8 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:09.094359 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:09.104913 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83c9bd-47fd-4252-8b1a-14b22c850ca8 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:09.106263 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796ef45-6d5c-4e06-b545-5de73b87bdc7 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:09.145087 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db61194-3a4c-4fc5-a72d-6f7c0b7c6192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:09.145714 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db61194-3a4c-4fc5-a72d-6f7c0b7c6192 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:09.146056 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db61194-3a4c-4fc5-a72d-6f7c0b7c6192 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:09.146335 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db61194-3a4c-4fc5-a72d-6f7c0b7c6192 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:09.147036 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db61194-3a4c-4fc5-a72d-6f7c0b7c6192 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:09.147336 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db61194-3a4c-4fc5-a72d-6f7c0b7c6192 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:09.148706 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db61194-3a4c-4fc5-a72d-6f7c0b7c6192 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:09.163221 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2708c4-fb4e-4594-b96c-3067a3fd18da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:09.163875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2708c4-fb4e-4594-b96c-3067a3fd18da None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:09.164154 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2708c4-fb4e-4594-b96c-3067a3fd18da None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:09.164425 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2708c4-fb4e-4594-b96c-3067a3fd18da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:09.176157 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 275/551] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:09.179888 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2708c4-fb4e-4594-b96c-3067a3fd18da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:09.181368 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2708c4-fb4e-4594-b96c-3067a3fd18da None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:09.182964 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db61194-3a4c-4fc5-a72d-6f7c0b7c6192 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:09.215049 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-6a2708c4-fb4e-4594-b96c-3067a3fd18da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:09.215049 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:09.215049 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:09.215049 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:09.215049 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:09.215049 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:09.215049 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:09.215049 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:09.215049 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:09.215049 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:09.215049 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:09.215049 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:09.215049 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:09.215049 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:09.215049 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:09.216037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:09.216037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:09.216037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:09.216037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:09.216037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:09.216037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:09.216037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:09.216037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:09.216037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:09.216037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:09.216037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:09.216037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:09.216037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:09.216037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:09.216037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:09.216037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:09.216037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:09.217035 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:09.217035 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:09.217035 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:09.217035 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:09.217035 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 277/552] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:09:09.223090 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2708c4-fb4e-4594-b96c-3067a3fd18da None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:09.281203 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:09.367805 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2708c4-fb4e-4594-b96c-3067a3fd18da None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:09.393284 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db61194-3a4c-4fc5-a72d-6f7c0b7c6192 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:09.430456 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e20eb63-67c7-46f8-aad3-47d62c7142b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:09.431081 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20eb63-67c7-46f8-aad3-47d62c7142b3 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:09.431363 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20eb63-67c7-46f8-aad3-47d62c7142b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:09.431640 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20eb63-67c7-46f8-aad3-47d62c7142b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:09.432352 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e20eb63-67c7-46f8-aad3-47d62c7142b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:09.432657 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e20eb63-67c7-46f8-aad3-47d62c7142b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:09.434036 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e20eb63-67c7-46f8-aad3-47d62c7142b3 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:09.461405 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c1d69d-cc6c-4a74-ba58-ecba33e5dadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:09.462074 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1d69d-cc6c-4a74-ba58-ecba33e5dadf None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:09.462354 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1d69d-cc6c-4a74-ba58-ecba33e5dadf None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:09.462626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1d69d-cc6c-4a74-ba58-ecba33e5dadf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:09.463187 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 278/553] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:09:09.465498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e20eb63-67c7-46f8-aad3-47d62c7142b3 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:09.479861 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c1d69d-cc6c-4a74-ba58-ecba33e5dadf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:09.481306 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c1d69d-cc6c-4a74-ba58-ecba33e5dadf None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:09.501717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-34c1d69d-cc6c-4a74-ba58-ecba33e5dadf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 04:09:09.501717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:09.501717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:09.501717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:09.501717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:09.501717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:09.501717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:09.501717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:09.501717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:09.501717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:09.501717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:09.501717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:09.501717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:09:09.501717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:09:09.501717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:09:09.501717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:09:09.502674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:09.502674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:09.502674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:09:09.502674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:09.502674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:09:09.502674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:09.502674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:09.502674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:09.502674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:09.502674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:09.502674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:09.502674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:09.502674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:09.502674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:09.502674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:09.502674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:09.502674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:09.502674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:09.503651 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:09.503651 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:09.503651 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:09:09.503651 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:09:09.503651 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:09:09.503651 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:09:09.503651 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:09:09.503651 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:09:09.503651 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 04:09:09.503651 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:09.504166 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 276/554] 10.222.0.46 () {56 vars in 1093 bytes} [Wed Jul 27 04:09:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:09:09.511316 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c1d69d-cc6c-4a74-ba58-ecba33e5dadf None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:09.526335 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:09.661006 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e20eb63-67c7-46f8-aad3-47d62c7142b3 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:09.673670 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:09.702442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e160d63-4438-4fa5-9d75-119b9ad69094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:09.703319 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e160d63-4438-4fa5-9d75-119b9ad69094 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:09.703613 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e160d63-4438-4fa5-9d75-119b9ad69094 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:09.703885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e160d63-4438-4fa5-9d75-119b9ad69094 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:09.713004 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e160d63-4438-4fa5-9d75-119b9ad69094 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:09.714631 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e160d63-4438-4fa5-9d75-119b9ad69094 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:09.716246 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c1d69d-cc6c-4a74-ba58-ecba33e5dadf None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:09.727919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-9e160d63-4438-4fa5-9d75-119b9ad69094 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 04:09:09.727919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:09.727919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:09.727919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:09.727919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:09.727919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:09.727919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:09.727919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:09.727919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:09.727919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:09.727919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:09.727919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:09.727919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:09:09.727919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:09:09.727919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:09:09.727919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:09:09.728873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:09.728873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:09.728873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:09:09.728873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:09.728873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:09:09.728873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:09.728873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:09.728873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:09.728873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:09.728873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:09.728873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:09.728873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:09.728873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:09.728873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:09.728873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:09.728873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:09.728873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:09.728873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:09.730346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:09.730346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:09.730346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:09:09.730346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:09:09.730346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:09:09.730346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:09:09.730346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:09:09.730346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:09:09.730346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 04:09:09.730346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:09.730346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 279/555] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:09:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:09.737107 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e160d63-4438-4fa5-9d75-119b9ad69094 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:09.818697 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f509b2-6ac0-48f6-a697-aec42ece3567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:09.819337 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f509b2-6ac0-48f6-a697-aec42ece3567 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:09.819613 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f509b2-6ac0-48f6-a697-aec42ece3567 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:09.819881 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f509b2-6ac0-48f6-a697-aec42ece3567 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:09.820941 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f509b2-6ac0-48f6-a697-aec42ece3567 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:09.821051 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f509b2-6ac0-48f6-a697-aec42ece3567 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:09.822439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f509b2-6ac0-48f6-a697-aec42ece3567 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:09.847032 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e160d63-4438-4fa5-9d75-119b9ad69094 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:09.866155 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 277/556] 10.222.0.46 () {56 vars in 1133 bytes} [Wed Jul 27 04:09:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:09.867050 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f509b2-6ac0-48f6-a697-aec42ece3567 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:09.887379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2f206-1444-42e1-bb75-61f7f9c85506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:09.888346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2f206-1444-42e1-bb75-61f7f9c85506 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:09.888832 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2f206-1444-42e1-bb75-61f7f9c85506 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:09.889310 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2f206-1444-42e1-bb75-61f7f9c85506 None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:09.890356 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d2f206-1444-42e1-bb75-61f7f9c85506 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:09.890867 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d2f206-1444-42e1-bb75-61f7f9c85506 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:09.892660 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d2f206-1444-42e1-bb75-61f7f9c85506 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:09.911307 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 280/557] 10.222.0.46 () {56 vars in 1135 bytes} [Wed Jul 27 04:09:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:09.917313 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2f206-1444-42e1-bb75-61f7f9c85506 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:09.945789 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:09.992611 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:10.022846 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2f206-1444-42e1-bb75-61f7f9c85506 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:10.053724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f509b2-6ac0-48f6-a697-aec42ece3567 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:10.058215 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6dea28-acac-4d94-a0fd-46174df98e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:10.058951 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6dea28-acac-4d94-a0fd-46174df98e12 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:10.059339 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6dea28-acac-4d94-a0fd-46174df98e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:10.059713 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6dea28-acac-4d94-a0fd-46174df98e12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:10.122714 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a348ed-3d94-4263-8c0a-d5ba5d543fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:10.123391 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a348ed-3d94-4263-8c0a-d5ba5d543fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:10.123675 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a348ed-3d94-4263-8c0a-d5ba5d543fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:10.123952 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a348ed-3d94-4263-8c0a-d5ba5d543fc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:10.132215 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6dea28-acac-4d94-a0fd-46174df98e12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:10.133819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6dea28-acac-4d94-a0fd-46174df98e12 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:10.156396 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a348ed-3d94-4263-8c0a-d5ba5d543fc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:10.157913 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a348ed-3d94-4263-8c0a-d5ba5d543fc0 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:10.166430 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:10.186334 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-b6a348ed-3d94-4263-8c0a-d5ba5d543fc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:10.186334 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:10.186334 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:10.186334 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:10.186334 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:10.186334 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:10.186334 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:10.186334 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:10.186334 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:10.186334 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:10.186334 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:10.186334 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:10.186334 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:10.186334 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:10.186334 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:10.187620 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:10.187620 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:10.187620 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:10.187620 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:10.187620 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:10.187620 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:10.187620 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:10.187620 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:10.187620 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:10.187620 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:10.187620 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:10.187620 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:10.187620 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:10.187620 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:10.187620 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:10.187620 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:10.187620 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:10.189503 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:10.189503 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:10.189503 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:10.189503 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:10.194452 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 278/558] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:10.202983 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a348ed-3d94-4263-8c0a-d5ba5d543fc0 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:10.246989 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 281/559] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:09:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:09:10.252615 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6dea28-acac-4d94-a0fd-46174df98e12 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:10.355890 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a348ed-3d94-4263-8c0a-d5ba5d543fc0 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:10.370911 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6dea28-acac-4d94-a0fd-46174df98e12 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:10.413998 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:10.420470 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:10.436254 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc8385-efa7-48bc-b579-4b55b6901c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:10.437138 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc8385-efa7-48bc-b579-4b55b6901c7d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:10.437493 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc8385-efa7-48bc-b579-4b55b6901c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:10.437887 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc8385-efa7-48bc-b579-4b55b6901c7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:10.468291 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dc8385-efa7-48bc-b579-4b55b6901c7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:10.469892 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dc8385-efa7-48bc-b579-4b55b6901c7d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:10.480808 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2108cd-0c87-41b5-8108-8f7f54998696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:10.481524 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2108cd-0c87-41b5-8108-8f7f54998696 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:10.481880 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2108cd-0c87-41b5-8108-8f7f54998696 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:10.482260 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2108cd-0c87-41b5-8108-8f7f54998696 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:10.483048 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2108cd-0c87-41b5-8108-8f7f54998696 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:10.483420 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2108cd-0c87-41b5-8108-8f7f54998696 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:10.484902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2108cd-0c87-41b5-8108-8f7f54998696 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:10.511659 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 279/560] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:10.519200 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f0f99-5674-4f9f-93da-c8d80d0ffa3d None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:10.523727 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f0f99-5674-4f9f-93da-c8d80d0ffa3d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:10.524090 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f0f99-5674-4f9f-93da-c8d80d0ffa3d None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:10.525582 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 280/561] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:10 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:10.529615 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f0f99-5674-4f9f-93da-c8d80d0ffa3d None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:10.529732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-42dc8385-efa7-48bc-b579-4b55b6901c7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:10.529732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:10.529732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:10.529732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:10.529732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:10.529732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:10.529732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:10.529732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:10.529732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:10.529732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:10.529732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:10.529732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:10.529732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:10.529732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:10.529732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:10.530770 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:10.530770 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:10.530770 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:10.530770 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:10.530770 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:10.530770 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:10.530770 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:10.530770 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:10.530770 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:10.530770 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:10.530770 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:10.530770 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:10.530770 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:10.530770 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:10.530770 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:10.530770 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:10.530770 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:10.531645 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:10.531645 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:10.531645 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:10.531645 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:10.535138 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 282/562] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:09:10.536479 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fee8ab-0275-4cf8-99eb-60169c1ca54a None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:10.536765 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fee8ab-0275-4cf8-99eb-60169c1ca54a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:10.537042 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fee8ab-0275-4cf8-99eb-60169c1ca54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:10.577541 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-72fee8ab-0275-4cf8-99eb-60169c1ca54a None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:10.578766 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-72fee8ab-0275-4cf8-99eb-60169c1ca54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:10.717999 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 283/563] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:09:10.717999 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fee8ab-0275-4cf8-99eb-60169c1ca54a None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:10.725307 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f0f99-5674-4f9f-93da-c8d80d0ffa3d None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:10.766400 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045faba3-83db-4f1e-8a77-833e845832d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:10.767279 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045faba3-83db-4f1e-8a77-833e845832d2 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:10.767629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045faba3-83db-4f1e-8a77-833e845832d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:10.767843 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045faba3-83db-4f1e-8a77-833e845832d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:10.775282 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045faba3-83db-4f1e-8a77-833e845832d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:10.776798 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045faba3-83db-4f1e-8a77-833e845832d2 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:10.788680 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-045faba3-83db-4f1e-8a77-833e845832d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:10.788680 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:10.788680 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:10.788680 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:10.788680 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:10.788680 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:10.788680 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:10.788680 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:10.788680 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:10.788680 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:10.788680 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:10.788680 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:10.788680 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:09:10.788680 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:09:10.788680 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:09:10.789593 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:09:10.789593 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:10.789593 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:10.789593 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:10.789593 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:10.789593 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:10.789593 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:10.789593 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:10.789593 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:10.789593 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:10.789593 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:10.789593 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:10.789593 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:10.789593 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:10.789593 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:10.789593 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:10.789593 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:10.790513 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:09:10.790513 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:09:10.790513 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:09:10.790513 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:09:10.790513 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:09:10.790513 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:09:10.790513 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:10.790513 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:10.794351 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 281/564] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:10.795691 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca65cc-42d6-43ff-88ca-f5bc9eea17db None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:10.795979 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca65cc-42d6-43ff-88ca-f5bc9eea17db None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:10.796257 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca65cc-42d6-43ff-88ca-f5bc9eea17db None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:10.798787 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 282/565] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:10.799532 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca65cc-42d6-43ff-88ca-f5bc9eea17db None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:10.843752 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:10.845106 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fee8ab-0275-4cf8-99eb-60169c1ca54a None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:10.854538 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:10.880376 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd517ce-640b-464b-9b19-cd022e2080a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:10.881013 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd517ce-640b-464b-9b19-cd022e2080a1 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:10.881300 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd517ce-640b-464b-9b19-cd022e2080a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:10.881580 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd517ce-640b-464b-9b19-cd022e2080a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:10.882318 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd517ce-640b-464b-9b19-cd022e2080a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:10.882628 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd517ce-640b-464b-9b19-cd022e2080a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:10.884006 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd517ce-640b-464b-9b19-cd022e2080a1 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:10.895491 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 284/566] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:10.896275 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd517ce-640b-464b-9b19-cd022e2080a1 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:10.906960 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca65cc-42d6-43ff-88ca-f5bc9eea17db None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:10.942869 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196f4d7-c7a4-43d3-9b87-ddc87c735c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:10.943518 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196f4d7-c7a4-43d3-9b87-ddc87c735c50 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:10.943804 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196f4d7-c7a4-43d3-9b87-ddc87c735c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:10.944082 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196f4d7-c7a4-43d3-9b87-ddc87c735c50 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:10.954372 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196f4d7-c7a4-43d3-9b87-ddc87c735c50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:09:10.960588 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-f196f4d7-c7a4-43d3-9b87-ddc87c735c50 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:10.960588 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:10.960588 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:10.960588 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:10.960588 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:10.960588 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:10.960588 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:10.960588 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:10.960588 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:10.960588 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:10.960588 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:10.960588 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:10.960588 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:09:10.960588 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:09:10.960588 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:09:10.960588 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:09:10.961673 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:09:10.961673 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:09:10.961673 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:09:10.961673 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:09:10.961673 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:10.961673 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:10.961673 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:10.961673 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:10.961673 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:10.961673 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:10.961673 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:10.961673 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:10.961673 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:10.961673 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:10.961673 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:10.961673 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:10.962564 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:10.962564 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:10.962564 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:10.962564 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:10.962564 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:09:10.962564 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:09:10.962564 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:09:10.962564 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:09:10.962564 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:09:10.962564 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:09:10.962564 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:10.962564 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:10.962564 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 283/567] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 27 04:09:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:10.963251 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485fd48e-4a79-43e0-8033-d6dbb401dc53 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:10.963251 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485fd48e-4a79-43e0-8033-d6dbb401dc53 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:10.963419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485fd48e-4a79-43e0-8033-d6dbb401dc53 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:10.966016 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 284/568] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:10.966016 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01f57c-87ce-4912-b467-efb842a0c23d None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:10.966260 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01f57c-87ce-4912-b467-efb842a0c23d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:10.966496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01f57c-87ce-4912-b467-efb842a0c23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:10.984239 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-ae01f57c-87ce-4912-b467-efb842a0c23d None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:10.985998 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ae01f57c-87ce-4912-b467-efb842a0c23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:11.003097 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd517ce-640b-464b-9b19-cd022e2080a1 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:11.036691 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742aa92-4c10-4c8a-81cb-1e0640c27293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:11.037227 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742aa92-4c10-4c8a-81cb-1e0640c27293 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:11.037510 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742aa92-4c10-4c8a-81cb-1e0640c27293 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:11.037787 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742aa92-4c10-4c8a-81cb-1e0640c27293 None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:11.038582 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d742aa92-4c10-4c8a-81cb-1e0640c27293 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:11.038889 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d742aa92-4c10-4c8a-81cb-1e0640c27293 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:11.040314 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d742aa92-4c10-4c8a-81cb-1e0640c27293 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:11.051947 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 285/569] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:09:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:11.052741 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742aa92-4c10-4c8a-81cb-1e0640c27293 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:11.053093 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 285/570] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:09:11.053881 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01f57c-87ce-4912-b467-efb842a0c23d None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:11.082584 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:11.087838 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:11.154881 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01f57c-87ce-4912-b467-efb842a0c23d None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:11.162164 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742aa92-4c10-4c8a-81cb-1e0640c27293 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:11.189771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7caf93-6bb8-4110-8248-b42df8642b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:11.190427 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7caf93-6bb8-4110-8248-b42df8642b39 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:11.190713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7caf93-6bb8-4110-8248-b42df8642b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:11.190987 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7caf93-6bb8-4110-8248-b42df8642b39 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:11.191715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7caf93-6bb8-4110-8248-b42df8642b39 None admin] RBAC: Adding query filter params (name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:11.192018 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7caf93-6bb8-4110-8248-b42df8642b39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:11.193391 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7caf93-6bb8-4110-8248-b42df8642b39 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:11.197421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359d10c-8c37-4afe-af4e-42eb223859a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:11.198097 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359d10c-8c37-4afe-af4e-42eb223859a3 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:11.198808 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359d10c-8c37-4afe-af4e-42eb223859a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:11.199095 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359d10c-8c37-4afe-af4e-42eb223859a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:11.201342 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 286/571] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 27 04:09:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:11.202602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc77d81-65d4-41d6-b433-3d4cd664097f None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:11.202893 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc77d81-65d4-41d6-b433-3d4cd664097f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:11.203173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc77d81-65d4-41d6-b433-3d4cd664097f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:11.206418 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f359d10c-8c37-4afe-af4e-42eb223859a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:11.207875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f359d10c-8c37-4afe-af4e-42eb223859a3 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:11.220353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-f359d10c-8c37-4afe-af4e-42eb223859a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:11.220353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:11.220353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:11.220353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:11.220353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:11.220353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:11.220353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:11.220353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:11.220353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:11.220353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:11.220353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:11.220353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:11.220353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:09:11.220353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:09:11.220353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:09:11.221244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:09:11.221244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:11.221244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:11.221244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:11.221244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:11.221244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:11.221244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:11.221244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:11.221244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:11.221244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:11.221244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:11.221244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:11.221244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:11.221244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:11.221244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:11.221244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:11.221244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:11.222256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:09:11.222256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:09:11.222256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:09:11.222256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:09:11.222256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:09:11.222256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:09:11.222256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:11.222256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:11.222677 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-6bc77d81-65d4-41d6-b433-3d4cd664097f None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:11.222763 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 286/572] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:11.222834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-6bc77d81-65d4-41d6-b433-3d4cd664097f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:11.222903 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359d10c-8c37-4afe-af4e-42eb223859a3 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:11.289378 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 287/573] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:09:11.290154 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc77d81-65d4-41d6-b433-3d4cd664097f None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:11.309226 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:11.323371 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:11.323662 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359d10c-8c37-4afe-af4e-42eb223859a3 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:11.359925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82c3f6a-7917-43f3-a74d-5ce681260110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:11.360643 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c3f6a-7917-43f3-a74d-5ce681260110 None admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:11.360919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c3f6a-7917-43f3-a74d-5ce681260110 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:11.361189 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c3f6a-7917-43f3-a74d-5ce681260110 None admin] PATH_INFO: `/v3/projects/e646d320aa5a4f78bfccaf941cf46478/users/9dc9ea8f89c14a2a8e3272f750647dfd/roles/6521132834a64b029695bbdd5eb251d2` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:11.374975 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82c3f6a-7917-43f3-a74d-5ce681260110 None admin] RBAC: Authorizing `identity:create_grant(project_id=e646d320aa5a4f78bfccaf941cf46478, user_id=9dc9ea8f89c14a2a8e3272f750647dfd, role_id=6521132834a64b029695bbdd5eb251d2)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:11.376941 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82c3f6a-7917-43f3-a74d-5ce681260110 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:11.389419 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 287/574] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 27 04:09:11 2022] PUT /identity/v3/projects/e646d320aa5a4f78bfccaf941cf46478/users/9dc9ea8f89c14a2a8e3272f750647dfd/roles/6521132834a64b029695bbdd5eb251d2 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:09:11.390222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82c3f6a-7917-43f3-a74d-5ce681260110 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:11.402232 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc77d81-65d4-41d6-b433-3d4cd664097f None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:11.442181 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0a3d3-d9ec-476d-864a-aa671a3f6eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:11.442852 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0a3d3-d9ec-476d-864a-aa671a3f6eed None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:11.443130 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0a3d3-d9ec-476d-864a-aa671a3f6eed None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:11.443399 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0a3d3-d9ec-476d-864a-aa671a3f6eed None admin] PATH_INFO: `/v3/roles/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:11.458225 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b0a3d3-d9ec-476d-864a-aa671a3f6eed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:09:11.465476 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-81b0a3d3-d9ec-476d-864a-aa671a3f6eed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:11.465476 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:11.465476 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:11.465476 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:11.465476 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:11.465476 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:11.465476 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:11.465476 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:11.465476 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:11.465476 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:11.465476 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:11.465476 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:11.465476 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:09:11.465476 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:09:11.465476 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:09:11.465476 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:09:11.466780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:09:11.466780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:09:11.466780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:09:11.466780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:09:11.466780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:11.466780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:11.466780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:11.466780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:11.466780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:11.466780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:11.466780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:11.466780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:11.466780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:11.466780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:11.466780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:11.466780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:11.467604 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:11.467604 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:11.467604 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:11.467604 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:11.467604 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:09:11.467604 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:09:11.467604 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:09:11.467604 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:09:11.467604 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:09:11.467604 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:09:11.467604 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:11.467604 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:11.467604 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 288/575] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 27 04:09:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:11.468297 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0a3d3-d9ec-476d-864a-aa671a3f6eed None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:11.533572 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82c3f6a-7917-43f3-a74d-5ce681260110 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:11.568590 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:11.575116 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8770f7-5ae7-449b-b43e-4fe559c1011f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:11.575777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8770f7-5ae7-449b-b43e-4fe559c1011f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:11.576063 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8770f7-5ae7-449b-b43e-4fe559c1011f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:11.576612 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8770f7-5ae7-449b-b43e-4fe559c1011f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:11.584413 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:11.588832 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8770f7-5ae7-449b-b43e-4fe559c1011f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:11.589747 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0a3d3-d9ec-476d-864a-aa671a3f6eed None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:11.590893 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8770f7-5ae7-449b-b43e-4fe559c1011f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:11.608567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-8f8770f7-5ae7-449b-b43e-4fe559c1011f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:11.608567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:11.608567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:11.608567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:11.608567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:11.608567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:11.608567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:11.608567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:11.608567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:11.608567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:11.608567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:11.608567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:11.608567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:11.608567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:11.608567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:11.609663 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:11.609663 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:11.609663 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:11.609663 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:11.609663 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:11.609663 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:11.609663 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:11.609663 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:11.609663 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:11.609663 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:11.609663 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:11.609663 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:11.609663 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:11.609663 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:11.609663 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:11.609663 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:11.609663 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:11.611139 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:11.611139 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:11.611139 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:11.611139 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:11.611139 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 288/576] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:11.611139 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8770f7-5ae7-449b-b43e-4fe559c1011f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:11.632493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10bf2ba-72b3-4ed0-b3dc-8cf9c0d362cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:11.633243 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10bf2ba-72b3-4ed0-b3dc-8cf9c0d362cb None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:11.633615 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10bf2ba-72b3-4ed0-b3dc-8cf9c0d362cb None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:11.633957 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10bf2ba-72b3-4ed0-b3dc-8cf9c0d362cb None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:11.635127 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10bf2ba-72b3-4ed0-b3dc-8cf9c0d362cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:11.635127 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10bf2ba-72b3-4ed0-b3dc-8cf9c0d362cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:11.636619 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10bf2ba-72b3-4ed0-b3dc-8cf9c0d362cb None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:11.649052 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 289/577] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:09:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:11.650859 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10bf2ba-72b3-4ed0-b3dc-8cf9c0d362cb None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:11.719979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8770f7-5ae7-449b-b43e-4fe559c1011f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:11.757248 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21e9099-7f9b-46e5-9b75-ef40407181cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:11.757905 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e9099-7f9b-46e5-9b75-ef40407181cd None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:11.758216 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e9099-7f9b-46e5-9b75-ef40407181cd None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:11.758482 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e9099-7f9b-46e5-9b75-ef40407181cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:11.761727 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10bf2ba-72b3-4ed0-b3dc-8cf9c0d362cb None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:11.771405 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21e9099-7f9b-46e5-9b75-ef40407181cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:09:11.776656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-a21e9099-7f9b-46e5-9b75-ef40407181cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:11.776656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:11.776656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:11.776656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:11.776656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:11.776656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:11.776656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:11.776656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:11.776656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:11.776656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:11.776656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:11.776656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:11.776656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:09:11.776656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:09:11.776656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:09:11.776656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:09:11.777616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:09:11.777616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:09:11.777616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:09:11.777616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:09:11.777616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:11.777616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:11.777616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:11.777616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:11.777616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:11.777616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:11.777616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:11.777616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:11.777616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:11.777616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:11.777616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:11.777616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:11.778503 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:11.778503 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:11.778503 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:11.778503 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:11.778503 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:09:11.778503 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:09:11.778503 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:09:11.778503 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:09:11.778503 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:09:11.778503 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:09:11.778503 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:11.778503 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:11.778503 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 289/578] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 27 04:09:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:11.779200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21e9099-7f9b-46e5-9b75-ef40407181cd None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:11.803583 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e206ecd-6da8-4633-9ffe-01da24181ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:11.804216 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e206ecd-6da8-4633-9ffe-01da24181ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:11.804657 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e206ecd-6da8-4633-9ffe-01da24181ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:11.804992 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e206ecd-6da8-4633-9ffe-01da24181ab6 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:11.805875 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e206ecd-6da8-4633-9ffe-01da24181ab6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:11.806229 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e206ecd-6da8-4633-9ffe-01da24181ab6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:11.807644 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e206ecd-6da8-4633-9ffe-01da24181ab6 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:11.820703 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 290/579] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 27 04:09:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:11.820703 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e206ecd-6da8-4633-9ffe-01da24181ab6 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:11.823834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:11.834154 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:11.890821 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21e9099-7f9b-46e5-9b75-ef40407181cd None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:11.928047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc832de-a32a-45f3-a360-6ad2d07a2e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:11.928671 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc832de-a32a-45f3-a360-6ad2d07a2e4c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:11.928949 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc832de-a32a-45f3-a360-6ad2d07a2e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:11.929219 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc832de-a32a-45f3-a360-6ad2d07a2e4c None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:11.929540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e206ecd-6da8-4633-9ffe-01da24181ab6 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:11.929911 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc832de-a32a-45f3-a360-6ad2d07a2e4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:11.930242 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc832de-a32a-45f3-a360-6ad2d07a2e4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:11.931593 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc832de-a32a-45f3-a360-6ad2d07a2e4c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:11.944843 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 290/580] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:11.945675 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc832de-a32a-45f3-a360-6ad2d07a2e4c None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:11.971892 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18251f41-dc32-47bd-aa6a-1e1c24739f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:11.972511 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18251f41-dc32-47bd-aa6a-1e1c24739f33 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:11.972791 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18251f41-dc32-47bd-aa6a-1e1c24739f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:11.973061 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18251f41-dc32-47bd-aa6a-1e1c24739f33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:11.973903 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18251f41-dc32-47bd-aa6a-1e1c24739f33 None admin] RBAC: Adding query filter params (role.id=6521132834a64b029695bbdd5eb251d2, scope.project.id=e646d320aa5a4f78bfccaf941cf46478, user.id=57b5fa7517034a39930ce7c9d12e1875) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:11.974241 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18251f41-dc32-47bd-aa6a-1e1c24739f33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:11.975657 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18251f41-dc32-47bd-aa6a-1e1c24739f33 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:11.990017 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 291/581] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:09:11 2022] GET /identity/v3/role_assignments?user.id=57b5fa7517034a39930ce7c9d12e1875&scope.project.id=e646d320aa5a4f78bfccaf941cf46478&role.id=6521132834a64b029695bbdd5eb251d2 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:11.990017 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18251f41-dc32-47bd-aa6a-1e1c24739f33 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:12.063398 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc832de-a32a-45f3-a360-6ad2d07a2e4c None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:12.077466 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:12.086169 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:12.109072 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7cc570-aa6c-4e98-99ba-c15017c6b8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:12.110184 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7cc570-aa6c-4e98-99ba-c15017c6b8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:12.110865 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7cc570-aa6c-4e98-99ba-c15017c6b8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:12.111367 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7cc570-aa6c-4e98-99ba-c15017c6b8d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:12.112473 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7cc570-aa6c-4e98-99ba-c15017c6b8d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:12.113024 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7cc570-aa6c-4e98-99ba-c15017c6b8d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:12.114667 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7cc570-aa6c-4e98-99ba-c15017c6b8d3 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:12.125131 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 291/582] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 27 04:09:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:12.126972 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7cc570-aa6c-4e98-99ba-c15017c6b8d3 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:12.139813 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18251f41-dc32-47bd-aa6a-1e1c24739f33 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:12.200344 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4005f339-17f9-4c65-8f89-0c0d1065b4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:12.201172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005f339-17f9-4c65-8f89-0c0d1065b4a1 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:12.201453 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005f339-17f9-4c65-8f89-0c0d1065b4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:12.202044 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005f339-17f9-4c65-8f89-0c0d1065b4a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:12.217262 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4005f339-17f9-4c65-8f89-0c0d1065b4a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:12.219204 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4005f339-17f9-4c65-8f89-0c0d1065b4a1 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:12.241961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-4005f339-17f9-4c65-8f89-0c0d1065b4a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:12.241961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:12.241961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:12.241961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:12.241961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:12.241961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:12.241961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:12.241961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:12.241961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:12.241961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:12.241961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:12.241961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:12.241961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:12.241961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:12.241961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:12.242937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:12.242937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:12.242937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:12.242937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:12.242937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:12.242937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:12.242937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:12.242937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:12.242937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:12.242937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:12.242937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:12.242937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:12.242937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:12.242937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:12.242937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:12.242937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:12.242937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:12.243819 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:12.243819 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:12.243819 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:12.243819 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:12.243819 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 292/583] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:12.244106 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7cc570-aa6c-4e98-99ba-c15017c6b8d3 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:12.244363 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4005f339-17f9-4c65-8f89-0c0d1065b4a1 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:12.279814 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976dc85-609d-409f-85fe-c83f81eaabdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:12.280683 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976dc85-609d-409f-85fe-c83f81eaabdc None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:12.281056 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976dc85-609d-409f-85fe-c83f81eaabdc None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:12.281431 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976dc85-609d-409f-85fe-c83f81eaabdc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:12.294013 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b976dc85-609d-409f-85fe-c83f81eaabdc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:12.294013 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b976dc85-609d-409f-85fe-c83f81eaabdc None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:12.310452 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-b976dc85-609d-409f-85fe-c83f81eaabdc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 04:09:12.310452 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:12.310452 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:12.310452 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:12.310452 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:12.310452 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:12.310452 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:12.310452 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:12.310452 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:12.310452 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:12.310452 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:12.310452 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:12.310452 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:09:12.310452 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:09:12.310452 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:09:12.310452 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:09:12.311433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:12.311433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:12.311433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:09:12.311433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:12.311433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:09:12.311433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:12.311433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:12.311433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:12.311433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:12.311433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:12.311433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:12.311433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:12.311433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:12.311433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:12.311433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:12.311433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:12.311433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:12.311433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:12.312402 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:12.312402 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:12.312402 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:09:12.312402 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:09:12.312402 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:09:12.312402 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:09:12.312402 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:09:12.312402 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:09:12.312402 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 04:09:12.312402 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:12.320237 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 292/584] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:09:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:12.320237 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976dc85-609d-409f-85fe-c83f81eaabdc None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:12.351950 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:12.368998 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4005f339-17f9-4c65-8f89-0c0d1065b4a1 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:12.386575 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:12.405295 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6b1d5-7260-49ad-a7db-be98cabb7488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:12.406308 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6b1d5-7260-49ad-a7db-be98cabb7488 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:12.406816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6b1d5-7260-49ad-a7db-be98cabb7488 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:12.407372 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6b1d5-7260-49ad-a7db-be98cabb7488 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:12.418839 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6b1d5-7260-49ad-a7db-be98cabb7488 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:12.420531 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6b1d5-7260-49ad-a7db-be98cabb7488 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:12.430800 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976dc85-609d-409f-85fe-c83f81eaabdc None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:12.436168 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-bbc6b1d5-7260-49ad-a7db-be98cabb7488 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:12.436168 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:12.436168 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:12.436168 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:12.436168 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:12.436168 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:12.436168 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:12.436168 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:12.436168 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:12.436168 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:12.436168 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:12.436168 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:12.436168 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:12.436168 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:12.436168 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:12.437016 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:12.437016 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:12.437016 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:12.437016 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:12.437016 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:12.437016 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:12.437016 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:12.437016 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:12.437016 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:12.437016 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:12.437016 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:12.437016 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:12.437016 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:12.437016 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:12.437016 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:12.437016 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:12.437016 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:12.437891 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:12.437891 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:12.437891 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:12.437891 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:12.444603 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 293/585] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:12.444603 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6b1d5-7260-49ad-a7db-be98cabb7488 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:12.471278 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad70fcd9-ba40-450b-9142-d38ddce9da2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:12.472063 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad70fcd9-ba40-450b-9142-d38ddce9da2c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:12.472352 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad70fcd9-ba40-450b-9142-d38ddce9da2c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:12.472679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad70fcd9-ba40-450b-9142-d38ddce9da2c None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:12.473583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad70fcd9-ba40-450b-9142-d38ddce9da2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:12.473800 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad70fcd9-ba40-450b-9142-d38ddce9da2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:12.475244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad70fcd9-ba40-450b-9142-d38ddce9da2c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:12.495571 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 293/586] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:12.496535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad70fcd9-ba40-450b-9142-d38ddce9da2c None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:12.571221 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6b1d5-7260-49ad-a7db-be98cabb7488 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:12.602159 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:12.617997 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad70fcd9-ba40-450b-9142-d38ddce9da2c None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:12.618169 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e8c960-9860-449c-8792-b062c11157e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:12.618169 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8c960-9860-449c-8792-b062c11157e7 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:12.618169 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8c960-9860-449c-8792-b062c11157e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:12.618169 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8c960-9860-449c-8792-b062c11157e7 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:12.618473 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e8c960-9860-449c-8792-b062c11157e7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:12.618473 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e8c960-9860-449c-8792-b062c11157e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:12.618473 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e8c960-9860-449c-8792-b062c11157e7 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:12.636672 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 294/587] 10.222.0.46 () {56 vars in 1137 bytes} [Wed Jul 27 04:09:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:12.637628 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e8c960-9860-449c-8792-b062c11157e7 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:12.650341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13f477-1e76-4043-a07a-4641a6da5673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:12.650977 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13f477-1e76-4043-a07a-4641a6da5673 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:12.651264 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13f477-1e76-4043-a07a-4641a6da5673 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:12.651541 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13f477-1e76-4043-a07a-4641a6da5673 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:12.652241 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd13f477-1e76-4043-a07a-4641a6da5673 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:12.652550 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd13f477-1e76-4043-a07a-4641a6da5673 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:12.654173 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd13f477-1e76-4043-a07a-4641a6da5673 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:12.667284 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 294/588] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:12.667977 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:12.668814 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13f477-1e76-4043-a07a-4641a6da5673 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:12.744178 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e8c960-9860-449c-8792-b062c11157e7 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:12.778689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13f477-1e76-4043-a07a-4641a6da5673 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:12.780840 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301ee29-50cf-47e4-95f9-6cb2bcb634d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:12.781501 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301ee29-50cf-47e4-95f9-6cb2bcb634d6 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:12.781783 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301ee29-50cf-47e4-95f9-6cb2bcb634d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:12.782083 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301ee29-50cf-47e4-95f9-6cb2bcb634d6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:12.789869 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301ee29-50cf-47e4-95f9-6cb2bcb634d6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:12.791372 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301ee29-50cf-47e4-95f9-6cb2bcb634d6 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:12.803761 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-d301ee29-50cf-47e4-95f9-6cb2bcb634d6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 04:09:12.803761 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:12.803761 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:12.803761 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:12.803761 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:12.803761 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:12.803761 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:12.803761 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:12.803761 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:12.803761 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:12.803761 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:12.803761 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:12.803761 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:09:12.803761 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:09:12.803761 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:09:12.803761 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:09:12.804709 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:12.804709 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:12.804709 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:09:12.804709 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:12.804709 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:09:12.804709 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:12.804709 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:12.804709 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:12.804709 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:12.804709 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:12.804709 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:12.804709 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:12.804709 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:12.804709 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:12.804709 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:12.804709 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:12.804709 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:12.804709 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:12.805629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:12.805629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:12.805629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:09:12.805629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:09:12.805629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:09:12.805629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:09:12.805629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:09:12.805629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:09:12.805629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 04:09:12.805629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:12.809784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 295/589] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 27 04:09:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:12.810647 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301ee29-50cf-47e4-95f9-6cb2bcb634d6 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:12.815084 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36a1e02-2e39-4c27-8571-ad789c8fb0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:12.815942 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a1e02-2e39-4c27-8571-ad789c8fb0ca None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:12.816297 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a1e02-2e39-4c27-8571-ad789c8fb0ca None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:12.816646 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a1e02-2e39-4c27-8571-ad789c8fb0ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:12.828374 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36a1e02-2e39-4c27-8571-ad789c8fb0ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:12.830620 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36a1e02-2e39-4c27-8571-ad789c8fb0ca None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:12.846969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-b36a1e02-2e39-4c27-8571-ad789c8fb0ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:12.846969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:12.846969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:12.846969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:12.846969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:12.846969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:12.846969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:12.846969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:12.846969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:12.846969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:12.846969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:12.846969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:12.846969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:12.846969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:12.846969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:12.848766 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:12.848766 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:12.848766 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:12.848766 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:12.848766 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:12.848766 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:12.848766 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:12.848766 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:12.848766 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:12.848766 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:12.848766 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:12.848766 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:12.848766 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:12.848766 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:12.848766 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:12.848766 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:12.848766 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:12.850871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:12.850871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:12.850871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:12.850871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:12.850871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 295/590] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:12.850871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36a1e02-2e39-4c27-8571-ad789c8fb0ca None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:12.853958 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:12.906421 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:12.919449 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301ee29-50cf-47e4-95f9-6cb2bcb634d6 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:12.955355 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd93357-6f87-46cd-890d-1a545178dc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:12.956035 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd93357-6f87-46cd-890d-1a545178dc9d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:12.956301 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd93357-6f87-46cd-890d-1a545178dc9d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:12.956571 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd93357-6f87-46cd-890d-1a545178dc9d None admin] PATH_INFO: `/v3/users/glance` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:12.962310 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36a1e02-2e39-4c27-8571-ad789c8fb0ca None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:12.964458 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd93357-6f87-46cd-890d-1a545178dc9d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:12.965907 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd93357-6f87-46cd-890d-1a545178dc9d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:12.978163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-ebd93357-6f87-46cd-890d-1a545178dc9d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 27 04:09:12.978163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:12.978163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:12.978163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:12.978163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:12.978163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:12.978163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:12.978163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:12.978163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:12.978163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:12.978163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:12.978163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:12.978163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:09:12.978163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:09:12.978163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:09:12.978163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:09:12.979348 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:12.979348 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:12.979348 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:09:12.979348 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:12.979348 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:09:12.979348 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:12.979348 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:12.979348 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:12.979348 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:12.979348 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:12.979348 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:12.979348 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:12.979348 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:12.979348 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:12.979348 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:12.979348 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:12.979348 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:12.979348 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:12.980453 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:12.980453 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:12.980453 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:09:12.980453 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:09:12.980453 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:09:12.980453 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:09:12.980453 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:09:12.980453 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:09:12.980453 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 27 04:09:12.980453 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:12.980453 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 296/591] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:09:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:12.980453 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd93357-6f87-46cd-890d-1a545178dc9d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:13.003169 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149c19a-3fb9-4c72-bee5-89a0a3cf7140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:13.004167 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149c19a-3fb9-4c72-bee5-89a0a3cf7140 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:13.004598 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149c19a-3fb9-4c72-bee5-89a0a3cf7140 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:13.005052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149c19a-3fb9-4c72-bee5-89a0a3cf7140 None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:13.006051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4149c19a-3fb9-4c72-bee5-89a0a3cf7140 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:13.006603 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4149c19a-3fb9-4c72-bee5-89a0a3cf7140 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:13.008287 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4149c19a-3fb9-4c72-bee5-89a0a3cf7140 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:13.027750 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 296/592] 10.222.0.46 () {56 vars in 1139 bytes} [Wed Jul 27 04:09:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:13.028824 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149c19a-3fb9-4c72-bee5-89a0a3cf7140 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:13.102097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd93357-6f87-46cd-890d-1a545178dc9d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:13.106512 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:13.153623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149c19a-3fb9-4c72-bee5-89a0a3cf7140 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:13.154823 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d35d5-aabb-482b-b0f0-370475ba81ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:13.156715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d35d5-aabb-482b-b0f0-370475ba81ff None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:13.157181 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d35d5-aabb-482b-b0f0-370475ba81ff None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:13.158175 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d35d5-aabb-482b-b0f0-370475ba81ff None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:13.159029 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d35d5-aabb-482b-b0f0-370475ba81ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:13.159483 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d35d5-aabb-482b-b0f0-370475ba81ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:13.161018 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d35d5-aabb-482b-b0f0-370475ba81ff None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:13.178521 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 297/593] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:13.179843 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d35d5-aabb-482b-b0f0-370475ba81ff None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:13.193625 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806ea7a1-21d2-4bfc-9493-38f223a1d5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:13.194292 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ea7a1-21d2-4bfc-9493-38f223a1d5c9 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:13.194573 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ea7a1-21d2-4bfc-9493-38f223a1d5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:13.194806 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:13.194920 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ea7a1-21d2-4bfc-9493-38f223a1d5c9 None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:13.195662 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806ea7a1-21d2-4bfc-9493-38f223a1d5c9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:13.196183 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806ea7a1-21d2-4bfc-9493-38f223a1d5c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:13.197766 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806ea7a1-21d2-4bfc-9493-38f223a1d5c9 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:13.217560 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 297/594] 10.222.0.46 () {56 vars in 1137 bytes} [Wed Jul 27 04:09:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:13.218565 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806ea7a1-21d2-4bfc-9493-38f223a1d5c9 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:13.316632 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d35d5-aabb-482b-b0f0-370475ba81ff None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:13.335250 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806ea7a1-21d2-4bfc-9493-38f223a1d5c9 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:13.358803 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:13.367354 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc10356c-b28d-49ce-8bf9-59dd535b9537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:13.368308 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10356c-b28d-49ce-8bf9-59dd535b9537 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:13.368997 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10356c-b28d-49ce-8bf9-59dd535b9537 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:13.369425 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10356c-b28d-49ce-8bf9-59dd535b9537 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:13.374563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f215de-08ac-49cf-81ba-de765636fcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:13.375318 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f215de-08ac-49cf-81ba-de765636fcad None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:13.375701 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f215de-08ac-49cf-81ba-de765636fcad None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:13.376603 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f215de-08ac-49cf-81ba-de765636fcad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:13.382975 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc10356c-b28d-49ce-8bf9-59dd535b9537 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:13.384176 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f215de-08ac-49cf-81ba-de765636fcad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:13.385070 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc10356c-b28d-49ce-8bf9-59dd535b9537 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:13.385802 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f215de-08ac-49cf-81ba-de765636fcad None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:13.397621 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-d5f215de-08ac-49cf-81ba-de765636fcad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:13.397621 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:13.397621 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:13.397621 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:13.397621 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:13.397621 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:13.397621 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:13.397621 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:13.397621 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:13.397621 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:13.397621 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:13.397621 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:13.397621 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:09:13.397621 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:09:13.397621 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:09:13.398726 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:09:13.398726 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:13.398726 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:13.398726 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:13.398726 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:13.398726 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:13.398726 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:13.398726 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:13.398726 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:13.398726 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:13.398726 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:13.398726 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:13.398726 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:13.398726 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:13.398726 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:13.398726 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:13.398726 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:13.399826 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:09:13.399826 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:09:13.399826 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:09:13.399826 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:09:13.399826 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:09:13.399826 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:09:13.399826 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:13.399826 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:13.405237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-dc10356c-b28d-49ce-8bf9-59dd535b9537 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:13.405237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:13.405237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:13.405237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:13.405237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:13.405237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:13.405237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:13.405237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:13.405237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:13.405237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:13.405237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:13.405237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:13.405237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:13.405237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:13.405237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:13.406134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:13.406134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:13.406134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:13.406134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:13.406134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:13.406134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:13.406134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:13.406134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:13.406134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:13.406134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:13.406134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:13.406134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:13.406134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:13.406134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:13.406134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:13.406134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:13.406134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:13.406990 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:13.406990 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:13.406990 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:13.406990 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:13.407760 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 298/595] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:13.407760 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f215de-08ac-49cf-81ba-de765636fcad None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:13.409706 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 298/596] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:13.411048 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc10356c-b28d-49ce-8bf9-59dd535b9537 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:13.484531 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:13.514098 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f215de-08ac-49cf-81ba-de765636fcad None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:13.530269 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc10356c-b28d-49ce-8bf9-59dd535b9537 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:13.551528 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d4c50-8833-4fc6-b7fb-92abb4bfbfaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:13.552176 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d4c50-8833-4fc6-b7fb-92abb4bfbfaa None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:13.552465 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d4c50-8833-4fc6-b7fb-92abb4bfbfaa None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:13.552741 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d4c50-8833-4fc6-b7fb-92abb4bfbfaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:13.563993 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4d4c50-8833-4fc6-b7fb-92abb4bfbfaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:13.565028 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5812b3-f713-433b-932a-99c77cf5654b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:13.565448 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4d4c50-8833-4fc6-b7fb-92abb4bfbfaa None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:13.565637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5812b3-f713-433b-932a-99c77cf5654b None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:13.565918 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5812b3-f713-433b-932a-99c77cf5654b None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:13.566217 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5812b3-f713-433b-932a-99c77cf5654b None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:13.566969 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5812b3-f713-433b-932a-99c77cf5654b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:13.567280 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5812b3-f713-433b-932a-99c77cf5654b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:13.568698 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5812b3-f713-433b-932a-99c77cf5654b None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:13.581388 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-1c4d4c50-8833-4fc6-b7fb-92abb4bfbfaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:13.581388 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:13.581388 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:13.581388 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:13.581388 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:13.581388 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:13.581388 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:13.581388 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:13.581388 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:13.581388 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:13.581388 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:13.581388 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:13.581388 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:13.581388 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:13.581388 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:13.582315 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:13.582315 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:13.582315 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:13.582315 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:13.582315 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:13.582315 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:13.582315 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:13.582315 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:13.582315 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:13.582315 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:13.582315 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:13.582315 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:13.582315 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:13.582315 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:13.582315 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:13.582315 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:13.582315 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:13.583304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:13.583304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:13.583304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:13.583304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:13.583304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 299/597] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:13.584095 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d4c50-8833-4fc6-b7fb-92abb4bfbfaa None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:13.588748 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 299/598] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:09:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:13.590649 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e864c94-27f7-4771-a576-93dee0801bb4 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:13.591160 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e864c94-27f7-4771-a576-93dee0801bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:13.591645 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e864c94-27f7-4771-a576-93dee0801bb4 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:13.593666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 300/599] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:13.594635 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e864c94-27f7-4771-a576-93dee0801bb4 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:13.616716 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:13.687368 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d4c50-8833-4fc6-b7fb-92abb4bfbfaa None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:13.707538 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e864c94-27f7-4771-a576-93dee0801bb4 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:13.725908 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f187b5-6049-40cc-aad4-022dcc947c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:13.726577 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f187b5-6049-40cc-aad4-022dcc947c41 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:13.726861 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f187b5-6049-40cc-aad4-022dcc947c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:13.727137 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f187b5-6049-40cc-aad4-022dcc947c41 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:13.727642 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:13.728515 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f187b5-6049-40cc-aad4-022dcc947c41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:13.728910 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f187b5-6049-40cc-aad4-022dcc947c41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:13.730396 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f187b5-6049-40cc-aad4-022dcc947c41 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:13.742735 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bdbd3a-f182-431b-914e-dbf1cf759e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:13.743353 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdbd3a-f182-431b-914e-dbf1cf759e4b None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:13.748115 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdbd3a-f182-431b-914e-dbf1cf759e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:13.748270 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 300/600] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:13.748361 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdbd3a-f182-431b-914e-dbf1cf759e4b None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:13.748434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f187b5-6049-40cc-aad4-022dcc947c41 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:13.748503 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bdbd3a-f182-431b-914e-dbf1cf759e4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:13.748503 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bdbd3a-f182-431b-914e-dbf1cf759e4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:13.748503 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bdbd3a-f182-431b-914e-dbf1cf759e4b None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:13.762444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 301/601] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:13.763659 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057da062-2960-474f-a688-9c57cf57ae6b None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:13.763946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057da062-2960-474f-a688-9c57cf57ae6b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:13.764218 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057da062-2960-474f-a688-9c57cf57ae6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:13.783807 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-057da062-2960-474f-a688-9c57cf57ae6b None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:13.785562 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-057da062-2960-474f-a688-9c57cf57ae6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:13.852196 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:13.862787 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 302/602] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:09:13.863637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057da062-2960-474f-a688-9c57cf57ae6b None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:13.863895 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f187b5-6049-40cc-aad4-022dcc947c41 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:13.911189 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00097ff7-30d7-4a1c-90a5-25c2b3a56649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:13.911923 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00097ff7-30d7-4a1c-90a5-25c2b3a56649 None admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:13.912216 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00097ff7-30d7-4a1c-90a5-25c2b3a56649 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:13.912495 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00097ff7-30d7-4a1c-90a5-25c2b3a56649 None admin] PATH_INFO: `/v3/projects/e646d320aa5a4f78bfccaf941cf46478/users/134ef452d7814c73b7c371a1cdbcbc00/roles/6521132834a64b029695bbdd5eb251d2` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:13.928466 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00097ff7-30d7-4a1c-90a5-25c2b3a56649 None admin] RBAC: Authorizing `identity:create_grant(project_id=e646d320aa5a4f78bfccaf941cf46478, user_id=134ef452d7814c73b7c371a1cdbcbc00, role_id=6521132834a64b029695bbdd5eb251d2)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:13.929920 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00097ff7-30d7-4a1c-90a5-25c2b3a56649 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:13.954586 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 301/603] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 27 04:09:13 2022] PUT /identity/v3/projects/e646d320aa5a4f78bfccaf941cf46478/users/134ef452d7814c73b7c371a1cdbcbc00/roles/6521132834a64b029695bbdd5eb251d2 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:09:13.954586 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00097ff7-30d7-4a1c-90a5-25c2b3a56649 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:13.993211 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057da062-2960-474f-a688-9c57cf57ae6b None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:14.005471 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:14.033365 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c4d39-f274-4b4d-b8c8-a29d1b64307a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:14.034272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c4d39-f274-4b4d-b8c8-a29d1b64307a None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:14.034553 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c4d39-f274-4b4d-b8c8-a29d1b64307a None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:14.034838 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c4d39-f274-4b4d-b8c8-a29d1b64307a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:14.042311 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6c4d39-f274-4b4d-b8c8-a29d1b64307a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:14.043784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6c4d39-f274-4b4d-b8c8-a29d1b64307a None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:14.056946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-bf6c4d39-f274-4b4d-b8c8-a29d1b64307a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:14.056946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:14.056946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:14.056946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:14.056946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:14.056946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:14.056946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:14.056946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:14.056946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:14.056946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:14.056946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:14.056946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:14.056946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:09:14.056946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:09:14.056946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:09:14.057959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:09:14.057959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:14.057959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:14.057959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:14.057959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:14.057959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:14.057959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:14.057959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:14.057959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:14.057959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:14.057959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:14.057959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:14.057959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:14.057959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:14.057959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:14.057959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:14.057959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:14.059312 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:09:14.059312 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:09:14.059312 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:09:14.059312 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:09:14.059312 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:09:14.059312 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:09:14.059312 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:14.059312 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:14.059312 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 303/604] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:14.059312 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c4d39-f274-4b4d-b8c8-a29d1b64307a None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:14.078627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00097ff7-30d7-4a1c-90a5-25c2b3a56649 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:14.116431 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e88340-3f87-44ef-98ad-163bf887c5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:14.117090 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e88340-3f87-44ef-98ad-163bf887c5a5 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:14.117371 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e88340-3f87-44ef-98ad-163bf887c5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:14.117647 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e88340-3f87-44ef-98ad-163bf887c5a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:14.125213 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e88340-3f87-44ef-98ad-163bf887c5a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:14.127313 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e88340-3f87-44ef-98ad-163bf887c5a5 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:14.139493 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-83e88340-3f87-44ef-98ad-163bf887c5a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:14.139493 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:14.139493 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:14.139493 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:14.139493 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:14.139493 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:14.139493 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:14.139493 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:14.139493 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:14.139493 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:14.139493 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:14.139493 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:14.139493 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:09:14.139493 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:09:14.139493 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:09:14.140449 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:09:14.140449 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:14.140449 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:14.140449 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:14.140449 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:14.140449 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:14.140449 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:14.140449 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:14.140449 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:14.140449 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:14.140449 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:14.140449 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:14.140449 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:14.140449 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:14.140449 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:14.140449 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:14.140449 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:14.141337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:09:14.141337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:09:14.141337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:09:14.141337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:09:14.141337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:09:14.141337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:09:14.141337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:14.141337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:14.141337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 302/605] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:14.143067 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e88340-3f87-44ef-98ad-163bf887c5a5 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:14.150220 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:14.194245 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c4d39-f274-4b4d-b8c8-a29d1b64307a None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:14.229248 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4462eb-3a97-4080-8f9f-923206325948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:14.229897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4462eb-3a97-4080-8f9f-923206325948 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:14.230208 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4462eb-3a97-4080-8f9f-923206325948 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:14.230483 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4462eb-3a97-4080-8f9f-923206325948 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:14.241393 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4462eb-3a97-4080-8f9f-923206325948 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:09:14.246821 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-df4462eb-3a97-4080-8f9f-923206325948 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:14.246821 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:14.246821 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:14.246821 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:14.246821 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:14.246821 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:14.246821 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:14.246821 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:14.246821 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:14.246821 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:14.246821 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:14.246821 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:14.246821 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:09:14.246821 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:09:14.246821 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:09:14.246821 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:09:14.247749 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:09:14.247749 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:09:14.247749 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:09:14.247749 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:09:14.247749 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:14.247749 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:14.247749 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:14.247749 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:14.247749 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:14.247749 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:14.247749 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:14.247749 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:14.247749 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:14.247749 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:14.247749 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:14.247749 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:14.248596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:14.248596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:14.248596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:14.248596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:14.248596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:09:14.248596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:09:14.248596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:09:14.248596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:09:14.248596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:09:14.248596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:09:14.248596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:14.248596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:14.248596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 304/606] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 27 04:09:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:14.249448 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4462eb-3a97-4080-8f9f-923206325948 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:14.263748 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e88340-3f87-44ef-98ad-163bf887c5a5 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:14.283980 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:14.298602 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c8fea5-8ce3-4d72-9b63-595c45fc0ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:14.299231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8fea5-8ce3-4d72-9b63-595c45fc0ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:14.299512 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8fea5-8ce3-4d72-9b63-595c45fc0ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:14.299786 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8fea5-8ce3-4d72-9b63-595c45fc0ce0 None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:14.300557 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c8fea5-8ce3-4d72-9b63-595c45fc0ce0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:14.300850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c8fea5-8ce3-4d72-9b63-595c45fc0ce0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:14.302304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c8fea5-8ce3-4d72-9b63-595c45fc0ce0 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:14.315095 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 303/607] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:09:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:14.316258 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216769cc-f423-4773-99ae-4649dbc8cc1c None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:14.316546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216769cc-f423-4773-99ae-4649dbc8cc1c None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:14.316821 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216769cc-f423-4773-99ae-4649dbc8cc1c None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:14.318198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 304/608] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:14.318808 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216769cc-f423-4773-99ae-4649dbc8cc1c None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:14.370745 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4462eb-3a97-4080-8f9f-923206325948 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:14.381391 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:14.409357 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4ad7a-76e1-4e89-8e45-c6a36650943c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:14.410000 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4ad7a-76e1-4e89-8e45-c6a36650943c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:14.410287 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4ad7a-76e1-4e89-8e45-c6a36650943c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:14.410559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4ad7a-76e1-4e89-8e45-c6a36650943c None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:14.411427 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4ad7a-76e1-4e89-8e45-c6a36650943c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:14.411856 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4ad7a-76e1-4e89-8e45-c6a36650943c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:14.413296 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4ad7a-76e1-4e89-8e45-c6a36650943c None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:14.421562 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216769cc-f423-4773-99ae-4649dbc8cc1c None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:14.431764 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 305/609] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:09:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:14.431764 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdebd4da-6d3b-4e14-8053-6ad69515137b None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:14.431764 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdebd4da-6d3b-4e14-8053-6ad69515137b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:14.431764 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdebd4da-6d3b-4e14-8053-6ad69515137b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:14.448764 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-cdebd4da-6d3b-4e14-8053-6ad69515137b None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:14.449932 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-cdebd4da-6d3b-4e14-8053-6ad69515137b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:14.464070 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dcc3be-2a17-45b6-bc5f-a253d124fd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:14.464696 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcc3be-2a17-45b6-bc5f-a253d124fd8f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:14.465010 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcc3be-2a17-45b6-bc5f-a253d124fd8f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:14.467194 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcc3be-2a17-45b6-bc5f-a253d124fd8f None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:14.467194 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dcc3be-2a17-45b6-bc5f-a253d124fd8f None admin] RBAC: Adding query filter params (name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:14.467194 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dcc3be-2a17-45b6-bc5f-a253d124fd8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:14.467632 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dcc3be-2a17-45b6-bc5f-a253d124fd8f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:14.477082 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 305/610] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 27 04:09:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:14.477967 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dcc3be-2a17-45b6-bc5f-a253d124fd8f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:14.538914 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 306/611] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:09:14.539563 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdebd4da-6d3b-4e14-8053-6ad69515137b None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:14.561181 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:14.601779 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dcc3be-2a17-45b6-bc5f-a253d124fd8f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:14.631611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:14.643439 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd045b-2a6d-4c23-ab94-a1e41dd7137a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:14.644120 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd045b-2a6d-4c23-ab94-a1e41dd7137a None admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:14.644407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd045b-2a6d-4c23-ab94-a1e41dd7137a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:14.644684 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd045b-2a6d-4c23-ab94-a1e41dd7137a None admin] PATH_INFO: `/v3/projects/e646d320aa5a4f78bfccaf941cf46478/users/b4847f3736d844d78a206b7eab5f5fcd/roles/6521132834a64b029695bbdd5eb251d2` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:14.645048 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdebd4da-6d3b-4e14-8053-6ad69515137b None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:14.661860 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fd045b-2a6d-4c23-ab94-a1e41dd7137a None admin] RBAC: Authorizing `identity:create_grant(project_id=e646d320aa5a4f78bfccaf941cf46478, user_id=b4847f3736d844d78a206b7eab5f5fcd, role_id=6521132834a64b029695bbdd5eb251d2)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:14.663495 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fd045b-2a6d-4c23-ab94-a1e41dd7137a None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:14.679344 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 306/612] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 27 04:09:14 2022] PUT /identity/v3/projects/e646d320aa5a4f78bfccaf941cf46478/users/b4847f3736d844d78a206b7eab5f5fcd/roles/6521132834a64b029695bbdd5eb251d2 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:09:14.680175 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd045b-2a6d-4c23-ab94-a1e41dd7137a None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:14.694344 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c9bc47-c453-40a8-8d7f-683db4e1f461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:14.696238 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9bc47-c453-40a8-8d7f-683db4e1f461 None admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:14.696802 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9bc47-c453-40a8-8d7f-683db4e1f461 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:14.697288 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9bc47-c453-40a8-8d7f-683db4e1f461 None admin] PATH_INFO: `/v3/projects/e646d320aa5a4f78bfccaf941cf46478/users/3e98a41a0f304fc2bb36a18a77a2d729/roles/6521132834a64b029695bbdd5eb251d2` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:14.722311 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c9bc47-c453-40a8-8d7f-683db4e1f461 None admin] RBAC: Authorizing `identity:create_grant(project_id=e646d320aa5a4f78bfccaf941cf46478, user_id=3e98a41a0f304fc2bb36a18a77a2d729, role_id=6521132834a64b029695bbdd5eb251d2)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:14.725379 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c9bc47-c453-40a8-8d7f-683db4e1f461 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:14.746100 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 307/613] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 27 04:09:14 2022] PUT /identity/v3/projects/e646d320aa5a4f78bfccaf941cf46478/users/3e98a41a0f304fc2bb36a18a77a2d729/roles/6521132834a64b029695bbdd5eb251d2 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:09:14.747414 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c9bc47-c453-40a8-8d7f-683db4e1f461 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:14.826568 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd045b-2a6d-4c23-ab94-a1e41dd7137a None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:14.833488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:14.870399 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c9bc47-c453-40a8-8d7f-683db4e1f461 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:14.881471 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66255a-8afc-45dd-be93-a51f3f95ba00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:14.882718 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66255a-8afc-45dd-be93-a51f3f95ba00 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:14.883008 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66255a-8afc-45dd-be93-a51f3f95ba00 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:14.883287 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66255a-8afc-45dd-be93-a51f3f95ba00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:14.896656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e66255a-8afc-45dd-be93-a51f3f95ba00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:14.898237 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e66255a-8afc-45dd-be93-a51f3f95ba00 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:14.915467 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-1e66255a-8afc-45dd-be93-a51f3f95ba00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:14.915467 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:14.915467 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:14.915467 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:14.915467 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:14.915467 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:14.915467 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:14.915467 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:14.915467 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:14.915467 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:14.915467 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:14.915467 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:14.915467 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:14.915467 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:14.915467 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:14.916434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:14.916434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:14.916434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:14.916434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:14.916434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:14.916434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:14.916434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:14.916434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:14.916434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:14.916434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:14.916434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:14.916434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:14.916434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:14.916434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:14.916434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:14.916434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:14.916434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:14.917311 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:14.917311 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:14.917311 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:14.917311 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:14.917311 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 307/614] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:14.920534 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c96c5-97e6-4d00-b9e0-4bcab013d7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:14.921490 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c96c5-97e6-4d00-b9e0-4bcab013d7ec None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:14.921999 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c96c5-97e6-4d00-b9e0-4bcab013d7ec None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:14.922498 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c96c5-97e6-4d00-b9e0-4bcab013d7ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:14.922604 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66255a-8afc-45dd-be93-a51f3f95ba00 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:14.937857 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5c96c5-97e6-4d00-b9e0-4bcab013d7ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:09:14.945784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-0f5c96c5-97e6-4d00-b9e0-4bcab013d7ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:14.945784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:14.945784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:14.945784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:14.945784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:14.945784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:14.945784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:14.945784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:14.945784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:14.945784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:14.945784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:14.945784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:14.945784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:09:14.945784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:09:14.945784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:09:14.945784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:09:14.946947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:09:14.946947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:09:14.946947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:09:14.946947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:09:14.946947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:14.946947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:14.946947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:14.946947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:14.946947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:14.946947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:14.946947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:14.946947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:14.946947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:14.946947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:14.946947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:14.946947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:14.947961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:14.947961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:14.947961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:14.947961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:14.947961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:09:14.947961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:09:14.947961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:09:14.947961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:09:14.947961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:09:14.947961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:09:14.947961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:14.947961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:14.950795 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 308/615] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 27 04:09:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:14.956514 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c96c5-97e6-4d00-b9e0-4bcab013d7ec None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:14.960336 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:15.045125 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66255a-8afc-45dd-be93-a51f3f95ba00 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:15.086321 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c96c5-97e6-4d00-b9e0-4bcab013d7ec None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:15.094768 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187afb0-9bb3-4d8b-9954-8e05e5f5fb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:15.095938 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187afb0-9bb3-4d8b-9954-8e05e5f5fb1f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:15.096225 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187afb0-9bb3-4d8b-9954-8e05e5f5fb1f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:15.096505 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187afb0-9bb3-4d8b-9954-8e05e5f5fb1f None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:15.097214 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187afb0-9bb3-4d8b-9954-8e05e5f5fb1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:15.098005 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187afb0-9bb3-4d8b-9954-8e05e5f5fb1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:15.099793 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187afb0-9bb3-4d8b-9954-8e05e5f5fb1f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:15.117419 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 308/616] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:15.122593 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187afb0-9bb3-4d8b-9954-8e05e5f5fb1f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:15.134132 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551081d-b9c9-4b47-b8c9-c6a30d775344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:15.135002 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551081d-b9c9-4b47-b8c9-c6a30d775344 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:15.135452 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551081d-b9c9-4b47-b8c9-c6a30d775344 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:15.135859 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551081d-b9c9-4b47-b8c9-c6a30d775344 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:15.136730 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5551081d-b9c9-4b47-b8c9-c6a30d775344 None admin] RBAC: Adding query filter params (name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:15.137174 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5551081d-b9c9-4b47-b8c9-c6a30d775344 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:15.140517 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:15.143340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5551081d-b9c9-4b47-b8c9-c6a30d775344 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:15.155273 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 309/617] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 27 04:09:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:09:15.160747 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551081d-b9c9-4b47-b8c9-c6a30d775344 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:15.266234 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:15.285105 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187afb0-9bb3-4d8b-9954-8e05e5f5fb1f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:15.323038 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39edbe26-d2b9-4fdf-b5c6-5d9300d0ee09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:15.323789 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39edbe26-d2b9-4fdf-b5c6-5d9300d0ee09 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:15.324072 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39edbe26-d2b9-4fdf-b5c6-5d9300d0ee09 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:15.324644 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39edbe26-d2b9-4fdf-b5c6-5d9300d0ee09 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:15.332774 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39edbe26-d2b9-4fdf-b5c6-5d9300d0ee09 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:15.334801 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39edbe26-d2b9-4fdf-b5c6-5d9300d0ee09 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:15.348206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-39edbe26-d2b9-4fdf-b5c6-5d9300d0ee09 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 04:09:15.348206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:15.348206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:15.348206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:15.348206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:15.348206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:15.348206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:15.348206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:15.348206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:15.348206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:15.348206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:15.348206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:15.348206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:09:15.348206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:09:15.348206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:09:15.348206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:09:15.349145 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:15.349145 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:15.349145 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:09:15.349145 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:15.349145 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:09:15.349145 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:15.349145 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:15.349145 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:15.349145 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:15.349145 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:15.349145 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:15.349145 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:15.349145 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:15.349145 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:15.349145 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:15.349145 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:15.349145 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:15.349145 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:15.350117 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:15.350117 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:15.350117 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:09:15.350117 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:09:15.350117 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:09:15.350117 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:09:15.350117 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:09:15.350117 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:09:15.350117 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 04:09:15.350117 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:15.350117 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 309/618] 10.222.0.46 () {56 vars in 1093 bytes} [Wed Jul 27 04:09:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:09:15.355003 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39edbe26-d2b9-4fdf-b5c6-5d9300d0ee09 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:15.403414 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551081d-b9c9-4b47-b8c9-c6a30d775344 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:15.437884 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665cfd6-0563-43d7-aa62-285c848633b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:15.439418 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665cfd6-0563-43d7-aa62-285c848633b1 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:15.439697 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665cfd6-0563-43d7-aa62-285c848633b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:15.440085 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665cfd6-0563-43d7-aa62-285c848633b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:15.452069 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2665cfd6-0563-43d7-aa62-285c848633b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:15.453545 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2665cfd6-0563-43d7-aa62-285c848633b1 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:15.470779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-2665cfd6-0563-43d7-aa62-285c848633b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:15.470779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:15.470779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:15.470779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:15.470779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:15.470779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:15.470779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:15.470779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:15.470779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:15.470779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:15.470779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:15.470779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:15.470779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:15.470779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:15.470779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:15.471713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:15.471713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:15.471713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:15.471713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:15.471713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:15.471713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:15.471713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:15.471713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:15.471713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:15.471713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:15.471713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:15.471713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:15.471713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:15.471713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:15.471713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:15.471713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:15.471713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:15.472747 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:15.472747 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:15.472747 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:15.472747 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:15.472747 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 310/619] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:09:15.478603 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665cfd6-0563-43d7-aa62-285c848633b1 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:15.527575 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:15.574289 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39edbe26-d2b9-4fdf-b5c6-5d9300d0ee09 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:15.588246 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665cfd6-0563-43d7-aa62-285c848633b1 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:15.623454 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267eae9a-59e2-4cf3-8220-3e762a68fcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:15.624180 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267eae9a-59e2-4cf3-8220-3e762a68fcac None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:15.624462 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267eae9a-59e2-4cf3-8220-3e762a68fcac None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:15.624736 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267eae9a-59e2-4cf3-8220-3e762a68fcac None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:15.625436 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267eae9a-59e2-4cf3-8220-3e762a68fcac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:15.625736 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267eae9a-59e2-4cf3-8220-3e762a68fcac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:15.627854 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267eae9a-59e2-4cf3-8220-3e762a68fcac None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:15.640472 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 311/620] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:09:15.645764 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267eae9a-59e2-4cf3-8220-3e762a68fcac None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:15.647731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7fb3cf-f4f9-4d50-b545-580d04d014ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:15.648484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7fb3cf-f4f9-4d50-b545-580d04d014ff None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:15.648773 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7fb3cf-f4f9-4d50-b545-580d04d014ff None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:15.649052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7fb3cf-f4f9-4d50-b545-580d04d014ff None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:15.650004 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7fb3cf-f4f9-4d50-b545-580d04d014ff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:15.650317 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7fb3cf-f4f9-4d50-b545-580d04d014ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:15.651834 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7fb3cf-f4f9-4d50-b545-580d04d014ff None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:15.685924 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 310/621] 10.222.0.46 () {56 vars in 1133 bytes} [Wed Jul 27 04:09:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:15.692130 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7fb3cf-f4f9-4d50-b545-580d04d014ff None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:15.698361 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:15.753249 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267eae9a-59e2-4cf3-8220-3e762a68fcac None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:15.755667 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:15.790288 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2da8f1-eadf-4f7e-ba42-ff65ee34d4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:15.790933 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2da8f1-eadf-4f7e-ba42-ff65ee34d4f7 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:15.791218 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2da8f1-eadf-4f7e-ba42-ff65ee34d4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:15.791490 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2da8f1-eadf-4f7e-ba42-ff65ee34d4f7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:15.799554 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2da8f1-eadf-4f7e-ba42-ff65ee34d4f7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:15.801017 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2da8f1-eadf-4f7e-ba42-ff65ee34d4f7 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:15.813432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-4e2da8f1-eadf-4f7e-ba42-ff65ee34d4f7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 04:09:15.813432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:15.813432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:15.813432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:15.813432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:15.813432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:15.813432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:15.813432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:15.813432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:15.813432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:15.813432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:15.813432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:15.813432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:09:15.813432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:09:15.813432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:09:15.813432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:09:15.815635 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:15.815635 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:15.815635 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:09:15.815635 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:15.815635 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:09:15.815635 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:15.815635 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:15.815635 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:15.815635 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:15.815635 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:15.815635 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:15.815635 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:15.815635 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:15.815635 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:15.815635 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:15.815635 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:15.815635 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:15.815635 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:15.816961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:15.816961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:15.816961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:09:15.816961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:09:15.816961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:09:15.816961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:09:15.816961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:09:15.816961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:09:15.816961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 04:09:15.816961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:15.816961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 312/622] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:09:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:15.823769 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2da8f1-eadf-4f7e-ba42-ff65ee34d4f7 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:15.936258 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2da8f1-eadf-4f7e-ba42-ff65ee34d4f7 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:15.972335 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bbcea8-d3fb-4086-979a-70ae0973d647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:15.973090 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bbcea8-d3fb-4086-979a-70ae0973d647 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:15.973435 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bbcea8-d3fb-4086-979a-70ae0973d647 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:15.973774 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bbcea8-d3fb-4086-979a-70ae0973d647 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:15.974675 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bbcea8-d3fb-4086-979a-70ae0973d647 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:15.975312 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bbcea8-d3fb-4086-979a-70ae0973d647 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:15.976764 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bbcea8-d3fb-4086-979a-70ae0973d647 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:15.997223 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 313/623] 10.222.0.46 () {56 vars in 1135 bytes} [Wed Jul 27 04:09:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:09:16.002545 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bbcea8-d3fb-4086-979a-70ae0973d647 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:16.003943 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7fb3cf-f4f9-4d50-b545-580d04d014ff None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:16.034393 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:16.106464 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd82d9d-543a-4209-9838-ae5ca0d191a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:16.107098 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd82d9d-543a-4209-9838-ae5ca0d191a4 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:16.107382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd82d9d-543a-4209-9838-ae5ca0d191a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:16.107657 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd82d9d-543a-4209-9838-ae5ca0d191a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:16.154657 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bbcea8-d3fb-4086-979a-70ae0973d647 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:16.194194 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd82d9d-543a-4209-9838-ae5ca0d191a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:16.195592 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd82d9d-543a-4209-9838-ae5ca0d191a4 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:16.198315 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:16.202668 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9cb28-dba9-4252-823e-1a7cb5bd4c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:16.203607 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9cb28-dba9-4252-823e-1a7cb5bd4c0c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:16.204264 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9cb28-dba9-4252-823e-1a7cb5bd4c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:16.204953 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9cb28-dba9-4252-823e-1a7cb5bd4c0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:16.217368 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9cb28-dba9-4252-823e-1a7cb5bd4c0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:16.219305 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9cb28-dba9-4252-823e-1a7cb5bd4c0c None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:16.235946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-d0d9cb28-dba9-4252-823e-1a7cb5bd4c0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:16.235946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:16.235946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:16.235946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:16.235946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:16.235946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:16.235946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:16.235946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:16.235946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:16.235946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:16.235946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:16.235946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:16.235946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:16.235946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:16.235946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:16.236867 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:16.236867 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:16.236867 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:16.236867 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:16.236867 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:16.236867 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:16.236867 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:16.236867 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:16.236867 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:16.236867 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:16.236867 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:16.236867 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:16.236867 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:16.236867 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:16.236867 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:16.236867 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:16.236867 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:16.237785 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:16.237785 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:16.237785 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:16.237785 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:16.240152 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 314/624] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:09:16.241816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45bd87-649b-4aa0-a7d9-eda884bce976 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:16.242356 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45bd87-649b-4aa0-a7d9-eda884bce976 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:16.242862 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45bd87-649b-4aa0-a7d9-eda884bce976 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:16.244791 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 315/625] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:16.245720 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed45bd87-649b-4aa0-a7d9-eda884bce976 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:16.306347 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 311/626] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:09:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 04:09:16.307688 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd8177-e6cf-40bf-a918-e4e7d664e89c None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:16.308034 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd8177-e6cf-40bf-a918-e4e7d664e89c None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:16.308252 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd8177-e6cf-40bf-a918-e4e7d664e89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:16.328102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-13cd8177-e6cf-40bf-a918-e4e7d664e89c None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:16.329821 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-13cd8177-e6cf-40bf-a918-e4e7d664e89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:16.358872 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed45bd87-649b-4aa0-a7d9-eda884bce976 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:16.370412 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:16.398742 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91375b5-2df3-42b2-beef-8023d2fbba6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:16.399734 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91375b5-2df3-42b2-beef-8023d2fbba6b None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:16.400243 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 312/627] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:09:16.400657 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91375b5-2df3-42b2-beef-8023d2fbba6b None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:16.402266 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd8177-e6cf-40bf-a918-e4e7d664e89c None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:16.402365 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91375b5-2df3-42b2-beef-8023d2fbba6b None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:16.402365 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91375b5-2df3-42b2-beef-8023d2fbba6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:16.402740 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91375b5-2df3-42b2-beef-8023d2fbba6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:16.404771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91375b5-2df3-42b2-beef-8023d2fbba6b None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:16.424528 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 316/628] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:16.426527 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91375b5-2df3-42b2-beef-8023d2fbba6b None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:16.443240 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:16.510996 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd8177-e6cf-40bf-a918-e4e7d664e89c None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:16.546684 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc079c-10f3-4adb-9053-87c54b23a505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:16.547344 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc079c-10f3-4adb-9053-87c54b23a505 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:16.547631 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc079c-10f3-4adb-9053-87c54b23a505 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:16.548554 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc079c-10f3-4adb-9053-87c54b23a505 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:16.549454 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91375b5-2df3-42b2-beef-8023d2fbba6b None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:16.559656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bc079c-10f3-4adb-9053-87c54b23a505 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:16.561225 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bc079c-10f3-4adb-9053-87c54b23a505 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:16.578282 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-b4bc079c-10f3-4adb-9053-87c54b23a505 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:16.578282 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:16.578282 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:16.578282 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:16.578282 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:16.578282 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:16.578282 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:16.578282 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:16.578282 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:16.578282 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:16.578282 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:16.578282 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:16.578282 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:16.578282 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:16.578282 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:16.579890 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:16.579890 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:16.579890 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:16.579890 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:16.579890 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:16.579890 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:16.579890 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:16.579890 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:16.579890 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:16.579890 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:16.579890 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:16.579890 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:16.579890 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:16.579890 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:16.579890 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:16.579890 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:16.579890 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:16.581546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:16.581546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:16.581546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:16.581546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:16.584792 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 313/629] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:16.585640 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc079c-10f3-4adb-9053-87c54b23a505 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:16.590532 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ba6cb-585f-4818-87c3-3eb75c749973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:16.591182 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ba6cb-585f-4818-87c3-3eb75c749973 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:16.591460 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ba6cb-585f-4818-87c3-3eb75c749973 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:16.591731 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ba6cb-585f-4818-87c3-3eb75c749973 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:16.593696 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:16.601205 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ba6cb-585f-4818-87c3-3eb75c749973 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:16.603079 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ba6cb-585f-4818-87c3-3eb75c749973 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:16.619095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-1a9ba6cb-585f-4818-87c3-3eb75c749973 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:16.619095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:16.619095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:16.619095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:16.619095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:16.619095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:16.619095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:16.619095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:16.619095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:16.619095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:16.619095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:16.619095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:16.619095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:09:16.619095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:09:16.619095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:09:16.621071 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:09:16.621071 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:16.621071 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:16.621071 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:16.621071 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:16.621071 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:16.621071 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:16.621071 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:16.621071 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:16.621071 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:16.621071 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:16.621071 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:16.621071 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:16.621071 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:16.621071 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:16.621071 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:16.621071 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:16.623374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:09:16.623374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:09:16.623374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:09:16.623374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:09:16.623374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:09:16.623374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:09:16.623374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:16.623374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:16.623374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 317/630] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:09:16.623374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ba6cb-585f-4818-87c3-3eb75c749973 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:16.684035 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:16.701263 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc079c-10f3-4adb-9053-87c54b23a505 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:16.738055 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249ba25-947e-4207-8ffa-373931598847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:16.738644 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249ba25-947e-4207-8ffa-373931598847 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:16.738929 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249ba25-947e-4207-8ffa-373931598847 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:16.739206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249ba25-947e-4207-8ffa-373931598847 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:16.750586 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8249ba25-947e-4207-8ffa-373931598847 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:09:16.757812 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-8249ba25-947e-4207-8ffa-373931598847 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:16.757812 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:16.757812 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:16.757812 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:16.757812 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:16.757812 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:16.757812 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:16.757812 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:16.757812 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:16.757812 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:16.757812 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:16.757812 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:16.757812 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:09:16.757812 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:09:16.757812 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:09:16.757812 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:09:16.759679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:09:16.759679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:09:16.759679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:09:16.759679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:09:16.759679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:16.759679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:16.759679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:16.759679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:16.759679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:16.759679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:16.759679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:16.759679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:16.759679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:16.759679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:16.759679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:16.759679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:16.761563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:16.761563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:16.761563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:16.761563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:16.761563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:09:16.761563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:09:16.761563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:09:16.761563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:09:16.761563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:09:16.761563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:09:16.761563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:16.761563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:16.763083 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ba6cb-585f-4818-87c3-3eb75c749973 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:16.766172 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 314/631] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 27 04:09:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:16.766172 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249ba25-947e-4207-8ffa-373931598847 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:16.806528 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d190619-2229-476b-ba80-3c645c4ae43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:16.807410 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d190619-2229-476b-ba80-3c645c4ae43c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:16.807906 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d190619-2229-476b-ba80-3c645c4ae43c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:16.808462 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d190619-2229-476b-ba80-3c645c4ae43c None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:16.809463 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d190619-2229-476b-ba80-3c645c4ae43c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:16.809920 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d190619-2229-476b-ba80-3c645c4ae43c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:16.811482 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d190619-2229-476b-ba80-3c645c4ae43c None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:16.831900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 318/632] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:16.833101 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d190619-2229-476b-ba80-3c645c4ae43c None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:16.876052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249ba25-947e-4207-8ffa-373931598847 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:16.896281 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:16.912424 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd83a8d-ac66-404f-a5b0-5df8f347b204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:16.913053 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd83a8d-ac66-404f-a5b0-5df8f347b204 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:16.913335 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd83a8d-ac66-404f-a5b0-5df8f347b204 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:16.913609 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd83a8d-ac66-404f-a5b0-5df8f347b204 None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:16.914496 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd83a8d-ac66-404f-a5b0-5df8f347b204 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:16.914806 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd83a8d-ac66-404f-a5b0-5df8f347b204 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:16.916330 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd83a8d-ac66-404f-a5b0-5df8f347b204 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:16.929747 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 315/633] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:09:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:16.930714 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd83a8d-ac66-404f-a5b0-5df8f347b204 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:16.939785 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:16.950114 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d190619-2229-476b-ba80-3c645c4ae43c None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:16.989233 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9962e94-ccd1-4d7c-8e6c-e6a00aac0e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:16.990224 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9962e94-ccd1-4d7c-8e6c-e6a00aac0e67 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:16.990737 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9962e94-ccd1-4d7c-8e6c-e6a00aac0e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:16.991213 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9962e94-ccd1-4d7c-8e6c-e6a00aac0e67 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:16.992358 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9962e94-ccd1-4d7c-8e6c-e6a00aac0e67 None admin] RBAC: Adding query filter params (name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:16.992913 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9962e94-ccd1-4d7c-8e6c-e6a00aac0e67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:16.994611 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9962e94-ccd1-4d7c-8e6c-e6a00aac0e67 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:17.008773 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 319/634] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 27 04:09:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:17.010001 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9962e94-ccd1-4d7c-8e6c-e6a00aac0e67 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:17.038374 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd83a8d-ac66-404f-a5b0-5df8f347b204 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:17.074946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45ebf7b-647b-4af1-86f3-48fa9a42a17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:17.075626 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ebf7b-647b-4af1-86f3-48fa9a42a17b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:17.076043 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ebf7b-647b-4af1-86f3-48fa9a42a17b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:17.076275 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ebf7b-647b-4af1-86f3-48fa9a42a17b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:17.084265 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ebf7b-647b-4af1-86f3-48fa9a42a17b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:17.085835 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ebf7b-647b-4af1-86f3-48fa9a42a17b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:17.098306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-a45ebf7b-647b-4af1-86f3-48fa9a42a17b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:17.098306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:17.098306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:17.098306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:17.098306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:17.098306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:17.098306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:17.098306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:17.098306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:17.098306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:17.098306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:17.098306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:17.098306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:09:17.098306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:09:17.098306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:09:17.099271 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:09:17.099271 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:17.099271 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:17.099271 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:17.099271 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:17.099271 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:17.099271 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:17.099271 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:17.099271 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:17.099271 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:17.099271 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:17.099271 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:17.099271 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:17.099271 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:17.099271 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:17.099271 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:17.099271 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:17.100158 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:09:17.100158 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:09:17.100158 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:09:17.100158 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:09:17.100158 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:09:17.100158 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:09:17.100158 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:17.100158 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:17.100158 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 316/635] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:17.100642 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45ebf7b-647b-4af1-86f3-48fa9a42a17b None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:17.137115 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9962e94-ccd1-4d7c-8e6c-e6a00aac0e67 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:17.155884 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:17.173028 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:17.175053 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46db8ff-6fa1-4f52-b0d2-b5376ba338c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:17.175740 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46db8ff-6fa1-4f52-b0d2-b5376ba338c8 None admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:17.176038 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46db8ff-6fa1-4f52-b0d2-b5376ba338c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:17.176312 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46db8ff-6fa1-4f52-b0d2-b5376ba338c8 None admin] PATH_INFO: `/v3/projects/e646d320aa5a4f78bfccaf941cf46478/users/57b5fa7517034a39930ce7c9d12e1875/roles/6521132834a64b029695bbdd5eb251d2` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:17.192079 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46db8ff-6fa1-4f52-b0d2-b5376ba338c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=e646d320aa5a4f78bfccaf941cf46478, user_id=57b5fa7517034a39930ce7c9d12e1875, role_id=6521132834a64b029695bbdd5eb251d2)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:17.193541 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46db8ff-6fa1-4f52-b0d2-b5376ba338c8 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:17.214027 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 320/636] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 27 04:09:17 2022] PUT /identity/v3/projects/e646d320aa5a4f78bfccaf941cf46478/users/57b5fa7517034a39930ce7c9d12e1875/roles/6521132834a64b029695bbdd5eb251d2 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:09:17.214027 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46db8ff-6fa1-4f52-b0d2-b5376ba338c8 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:17.217852 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45ebf7b-647b-4af1-86f3-48fa9a42a17b None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:17.269536 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c9e356-0dea-4a5c-a837-c6aa45cc9f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:17.270234 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9e356-0dea-4a5c-a837-c6aa45cc9f38 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:17.270520 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9e356-0dea-4a5c-a837-c6aa45cc9f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:17.271052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9e356-0dea-4a5c-a837-c6aa45cc9f38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:17.287308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c9e356-0dea-4a5c-a837-c6aa45cc9f38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:17.289110 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c9e356-0dea-4a5c-a837-c6aa45cc9f38 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:17.305670 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-f7c9e356-0dea-4a5c-a837-c6aa45cc9f38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:17.305670 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:17.305670 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:17.305670 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:17.305670 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:17.305670 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:17.305670 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:17.305670 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:17.305670 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:17.305670 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:17.305670 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:17.305670 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:17.305670 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:17.305670 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:17.305670 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:17.306841 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:17.306841 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:17.306841 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:17.306841 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:17.306841 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:17.306841 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:17.306841 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:17.306841 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:17.306841 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:17.306841 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:17.306841 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:17.306841 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:17.306841 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:17.306841 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:17.306841 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:17.306841 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:17.306841 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:17.307938 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:17.307938 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:17.307938 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:17.307938 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:17.311554 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 317/637] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:17.313085 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef85ca-b07d-49a6-9bd1-9211a288e943 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:17.314335 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef85ca-b07d-49a6-9bd1-9211a288e943 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:17.314745 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef85ca-b07d-49a6-9bd1-9211a288e943 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:17.316102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 318/638] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:17.317191 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33943241-9584-4b22-836d-f67c7aca8e7e None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:17.317476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33943241-9584-4b22-836d-f67c7aca8e7e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:17.317750 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33943241-9584-4b22-836d-f67c7aca8e7e None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:17.319232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 319/639] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:17.319964 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33943241-9584-4b22-836d-f67c7aca8e7e None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:17.342244 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46db8ff-6fa1-4f52-b0d2-b5376ba338c8 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:17.393537 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8bdb9d-7abb-403f-b15f-b52129eaca29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:17.394233 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bdb9d-7abb-403f-b15f-b52129eaca29 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:17.394999 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bdb9d-7abb-403f-b15f-b52129eaca29 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:17.395279 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bdb9d-7abb-403f-b15f-b52129eaca29 None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:17.396311 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8bdb9d-7abb-403f-b15f-b52129eaca29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:17.396616 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8bdb9d-7abb-403f-b15f-b52129eaca29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:17.398504 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8bdb9d-7abb-403f-b15f-b52129eaca29 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:17.419725 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 321/640] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:09:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:17.419725 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b006fa6-b7b0-4e39-9fdb-7b1f1b50a008 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:17.419725 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b006fa6-b7b0-4e39-9fdb-7b1f1b50a008 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:17.419725 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b006fa6-b7b0-4e39-9fdb-7b1f1b50a008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:17.431933 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33943241-9584-4b22-836d-f67c7aca8e7e None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:17.439937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-4b006fa6-b7b0-4e39-9fdb-7b1f1b50a008 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:17.441508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-4b006fa6-b7b0-4e39-9fdb-7b1f1b50a008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:17.449115 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:17.468690 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1f0e6-0b73-46bb-aba4-aaf1b8d5b31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:17.469319 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1f0e6-0b73-46bb-aba4-aaf1b8d5b31d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:17.470155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1f0e6-0b73-46bb-aba4-aaf1b8d5b31d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:17.470439 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1f0e6-0b73-46bb-aba4-aaf1b8d5b31d None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:17.471154 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db1f0e6-0b73-46bb-aba4-aaf1b8d5b31d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:17.471458 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db1f0e6-0b73-46bb-aba4-aaf1b8d5b31d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:17.471698 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:17.472880 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db1f0e6-0b73-46bb-aba4-aaf1b8d5b31d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:17.493204 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 320/641] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:17.494944 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6516ef9-ef23-440a-bbf6-5b7eabcee4ad None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:17.495446 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6516ef9-ef23-440a-bbf6-5b7eabcee4ad None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:17.495908 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6516ef9-ef23-440a-bbf6-5b7eabcee4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:17.514288 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-d6516ef9-ef23-440a-bbf6-5b7eabcee4ad None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:17.515615 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d6516ef9-ef23-440a-bbf6-5b7eabcee4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:17.535367 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 322/642] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:09:17.543144 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b006fa6-b7b0-4e39-9fdb-7b1f1b50a008 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:17.586362 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 321/643] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:09:17.587210 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6516ef9-ef23-440a-bbf6-5b7eabcee4ad None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:17.677071 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b006fa6-b7b0-4e39-9fdb-7b1f1b50a008 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:17.699723 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6516ef9-ef23-440a-bbf6-5b7eabcee4ad None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:17.710166 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:17.715077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a59498b-580a-4506-accb-740c5ccab0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:17.715696 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59498b-580a-4506-accb-740c5ccab0be None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:17.715978 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59498b-580a-4506-accb-740c5ccab0be None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:17.716250 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59498b-580a-4506-accb-740c5ccab0be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:17.717041 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a59498b-580a-4506-accb-740c5ccab0be None admin] RBAC: Adding query filter params (role.id=6521132834a64b029695bbdd5eb251d2, scope.project.id=e646d320aa5a4f78bfccaf941cf46478, user.id=9dc9ea8f89c14a2a8e3272f750647dfd) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:17.717345 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a59498b-580a-4506-accb-740c5ccab0be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:17.718816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a59498b-580a-4506-accb-740c5ccab0be None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:17.730002 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 323/644] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:09:17 2022] GET /identity/v3/role_assignments?user.id=9dc9ea8f89c14a2a8e3272f750647dfd&scope.project.id=e646d320aa5a4f78bfccaf941cf46478&role.id=6521132834a64b029695bbdd5eb251d2 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:17.730002 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a59498b-580a-4506-accb-740c5ccab0be None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:17.738792 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af20b7-1442-436a-9843-b4409865c002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:17.739474 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af20b7-1442-436a-9843-b4409865c002 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:17.739760 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af20b7-1442-436a-9843-b4409865c002 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:17.740042 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af20b7-1442-436a-9843-b4409865c002 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:17.748687 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11af20b7-1442-436a-9843-b4409865c002 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:17.750243 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11af20b7-1442-436a-9843-b4409865c002 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:17.765007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-11af20b7-1442-436a-9843-b4409865c002 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 04:09:17.765007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:17.765007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:17.765007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:17.765007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:17.765007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:17.765007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:17.765007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:17.765007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:17.765007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:17.765007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:17.765007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:17.765007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:09:17.765007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:09:17.765007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:09:17.765007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:09:17.766185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:17.766185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:17.766185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:09:17.766185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:17.766185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:09:17.766185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:17.766185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:17.766185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:17.766185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:17.766185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:17.766185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:17.766185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:17.766185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:17.766185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:17.766185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:17.766185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:17.766185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:17.766185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:17.768442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:17.768442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:17.768442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:09:17.768442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:09:17.768442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:09:17.768442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:09:17.768442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:09:17.768442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:09:17.768442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 04:09:17.768442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:17.771638 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 322/645] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:09:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:17.772626 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af20b7-1442-436a-9843-b4409865c002 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:17.786556 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:17.896551 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a59498b-580a-4506-accb-740c5ccab0be None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:17.902139 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af20b7-1442-436a-9843-b4409865c002 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:17.944206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8ce6c-af01-4e67-8434-27ce7743bcf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:17.945020 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8ce6c-af01-4e67-8434-27ce7743bcf2 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:17.945312 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8ce6c-af01-4e67-8434-27ce7743bcf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:17.945577 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930456e9-7ce9-44bd-b417-f751d1ea0955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:17.945703 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8ce6c-af01-4e67-8434-27ce7743bcf2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:17.946310 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930456e9-7ce9-44bd-b417-f751d1ea0955 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:17.946549 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930456e9-7ce9-44bd-b417-f751d1ea0955 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:17.946822 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930456e9-7ce9-44bd-b417-f751d1ea0955 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:17.958216 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc8ce6c-af01-4e67-8434-27ce7743bcf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:09:17.959400 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930456e9-7ce9-44bd-b417-f751d1ea0955 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:09:17.963611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-0cc8ce6c-af01-4e67-8434-27ce7743bcf2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:17.963611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:17.963611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:17.963611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:17.963611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:17.963611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:17.963611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:17.963611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:17.963611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:17.963611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:17.963611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:17.963611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:17.963611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:09:17.963611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:09:17.963611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:09:17.963611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:09:17.964561 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:09:17.964561 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:09:17.964561 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:09:17.964561 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:09:17.964561 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:17.964561 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:17.964561 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:17.964561 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:17.964561 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:17.964561 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:17.964561 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:17.964561 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:17.964561 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:17.964561 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:17.964561 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:17.964561 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:17.965526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:17.965526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:17.965526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:17.965526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:17.965526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:09:17.965526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:09:17.965526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:09:17.965526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:09:17.965526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:09:17.965526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:09:17.965526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:17.965526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:17.965526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 323/646] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 27 04:09:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:17.966922 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8ce6c-af01-4e67-8434-27ce7743bcf2 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:17.967333 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-930456e9-7ce9-44bd-b417-f751d1ea0955 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:17.967333 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:17.967333 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:17.967333 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:17.967333 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:17.967333 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:17.967333 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:17.967333 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:17.967333 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:17.967333 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:17.967333 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:17.967333 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:17.967333 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:09:17.967333 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:09:17.967333 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:09:17.967333 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:09:17.968253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:09:17.968253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:09:17.968253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:09:17.968253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:09:17.968253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:17.968253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:17.968253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:17.968253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:17.968253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:17.968253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:17.968253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:17.968253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:17.968253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:17.968253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:17.968253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:17.968253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:17.969080 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:17.969080 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:17.969080 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:17.969080 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:17.969080 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:09:17.969080 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:09:17.969080 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:09:17.969080 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:09:17.969080 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:09:17.969080 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:09:17.969080 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:17.969080 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:17.972645 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 324/647] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 27 04:09:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:17.973658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930456e9-7ce9-44bd-b417-f751d1ea0955 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:17.998907 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:18.092326 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8ce6c-af01-4e67-8434-27ce7743bcf2 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:18.113206 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930456e9-7ce9-44bd-b417-f751d1ea0955 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:18.130937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:18.131488 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dba740-e8a7-4f3a-a0ec-910dc8f85a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:18.132810 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dba740-e8a7-4f3a-a0ec-910dc8f85a84 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:18.133049 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dba740-e8a7-4f3a-a0ec-910dc8f85a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:18.133338 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dba740-e8a7-4f3a-a0ec-910dc8f85a84 None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:18.134479 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dba740-e8a7-4f3a-a0ec-910dc8f85a84 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:18.134799 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dba740-e8a7-4f3a-a0ec-910dc8f85a84 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:18.137183 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dba740-e8a7-4f3a-a0ec-910dc8f85a84 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:18.159142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182298e-818d-47a0-86f4-310d0cd54af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:18.159804 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182298e-818d-47a0-86f4-310d0cd54af3 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:18.160094 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182298e-818d-47a0-86f4-310d0cd54af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:18.160371 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182298e-818d-47a0-86f4-310d0cd54af3 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:18.160758 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 324/648] 10.222.0.46 () {56 vars in 1137 bytes} [Wed Jul 27 04:09:17 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:18.161276 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e182298e-818d-47a0-86f4-310d0cd54af3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:18.161583 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e182298e-818d-47a0-86f4-310d0cd54af3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:18.161701 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dba740-e8a7-4f3a-a0ec-910dc8f85a84 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:18.167976 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e182298e-818d-47a0-86f4-310d0cd54af3 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:18.179414 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 325/649] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 27 04:09:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:18.180914 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182298e-818d-47a0-86f4-310d0cd54af3 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:18.284284 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:18.300233 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dba740-e8a7-4f3a-a0ec-910dc8f85a84 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:18.306911 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182298e-818d-47a0-86f4-310d0cd54af3 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:18.339310 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c6233-5e7d-42ff-b1e9-a458a26a2e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:18.339948 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c6233-5e7d-42ff-b1e9-a458a26a2e0b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:18.340223 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c6233-5e7d-42ff-b1e9-a458a26a2e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:18.340497 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c6233-5e7d-42ff-b1e9-a458a26a2e0b None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:18.341228 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c6233-5e7d-42ff-b1e9-a458a26a2e0b None admin] RBAC: Adding query filter params (name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:18.341534 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c6233-5e7d-42ff-b1e9-a458a26a2e0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:18.343098 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c6233-5e7d-42ff-b1e9-a458a26a2e0b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:18.348835 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d187f6-d29c-4ade-82b3-d2169b777a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:18.349478 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d187f6-d29c-4ade-82b3-d2169b777a82 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:18.349756 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d187f6-d29c-4ade-82b3-d2169b777a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:18.350054 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d187f6-d29c-4ade-82b3-d2169b777a82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:18.357183 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 325/650] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 27 04:09:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:18.358102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c6233-5e7d-42ff-b1e9-a458a26a2e0b None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:18.366309 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:18.433763 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d187f6-d29c-4ade-82b3-d2169b777a82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:18.435292 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d187f6-d29c-4ade-82b3-d2169b777a82 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:18.463829 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c6233-5e7d-42ff-b1e9-a458a26a2e0b None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:18.500618 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac8607-9c77-49e3-9cbd-f0b6ed4c579e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:18.501483 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac8607-9c77-49e3-9cbd-f0b6ed4c579e None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:18.501701 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac8607-9c77-49e3-9cbd-f0b6ed4c579e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:18.501896 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac8607-9c77-49e3-9cbd-f0b6ed4c579e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:18.512667 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cac8607-9c77-49e3-9cbd-f0b6ed4c579e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:18.514142 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cac8607-9c77-49e3-9cbd-f0b6ed4c579e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:18.522845 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:18.529472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-0cac8607-9c77-49e3-9cbd-f0b6ed4c579e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:18.529472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:18.529472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:18.529472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:18.529472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:18.529472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:18.529472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:18.529472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:18.529472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:18.529472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:18.529472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:18.529472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:18.529472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:18.529472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:18.529472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:18.531144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:18.531144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:18.531144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:18.531144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:18.531144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:18.531144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:18.531144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:18.531144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:18.531144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:18.531144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:18.531144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:18.531144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:18.531144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:18.531144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:18.531144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:18.531144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:18.531144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:18.532892 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:18.532892 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:18.532892 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:18.532892 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:18.532892 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 326/651] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:18.532892 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac8607-9c77-49e3-9cbd-f0b6ed4c579e None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:18.566808 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 326/652] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:09:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:09:18.567757 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d187f6-d29c-4ade-82b3-d2169b777a82 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:18.616131 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:18.655234 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac8607-9c77-49e3-9cbd-f0b6ed4c579e None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:18.688034 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d187f6-d29c-4ade-82b3-d2169b777a82 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:18.704907 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f23454-267c-4fad-98cf-643b7da33883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:18.706428 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f23454-267c-4fad-98cf-643b7da33883 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:18.707124 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f23454-267c-4fad-98cf-643b7da33883 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:18.707585 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f23454-267c-4fad-98cf-643b7da33883 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:18.724387 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f23454-267c-4fad-98cf-643b7da33883 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:18.726285 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f23454-267c-4fad-98cf-643b7da33883 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:18.726577 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc819526-2169-4618-93c4-fa7b976eef91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:18.727373 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc819526-2169-4618-93c4-fa7b976eef91 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:18.727655 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc819526-2169-4618-93c4-fa7b976eef91 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:18.727926 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc819526-2169-4618-93c4-fa7b976eef91 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:18.728620 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc819526-2169-4618-93c4-fa7b976eef91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:18.728918 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc819526-2169-4618-93c4-fa7b976eef91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:18.730293 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc819526-2169-4618-93c4-fa7b976eef91 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:18.743093 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 327/653] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:18.744117 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc819526-2169-4618-93c4-fa7b976eef91 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:18.746191 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-39f23454-267c-4fad-98cf-643b7da33883 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:18.746191 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:18.746191 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:18.746191 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:18.746191 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:18.746191 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:18.746191 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:18.746191 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:18.746191 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:18.746191 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:18.746191 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:18.746191 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:18.746191 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:18.746191 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:18.746191 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:18.747308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:18.747308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:18.747308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:18.747308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:18.747308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:18.747308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:18.747308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:18.747308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:18.747308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:18.747308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:18.747308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:18.747308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:18.747308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:18.747308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:18.747308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:18.747308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:18.747308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:18.748250 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:18.748250 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:18.748250 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:18.748250 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:18.752413 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 327/654] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:18.753399 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f23454-267c-4fad-98cf-643b7da33883 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:18.829839 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:18.870001 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc819526-2169-4618-93c4-fa7b976eef91 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:18.871678 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:18.893192 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f23454-267c-4fad-98cf-643b7da33883 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:18.916350 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816d634-675f-41ef-b34b-eea868a1f742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:18.917494 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816d634-675f-41ef-b34b-eea868a1f742 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:18.918207 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816d634-675f-41ef-b34b-eea868a1f742 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:18.918505 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816d634-675f-41ef-b34b-eea868a1f742 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:18.932586 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5816d634-675f-41ef-b34b-eea868a1f742 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:18.934171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5816d634-675f-41ef-b34b-eea868a1f742 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:18.935413 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf9495-583f-469d-993b-5a7232f55736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:18.936584 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf9495-583f-469d-993b-5a7232f55736 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:18.937421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf9495-583f-469d-993b-5a7232f55736 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:18.937894 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf9495-583f-469d-993b-5a7232f55736 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:18.949173 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cf9495-583f-469d-993b-5a7232f55736 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:18.950644 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cf9495-583f-469d-993b-5a7232f55736 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:18.955750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-5816d634-675f-41ef-b34b-eea868a1f742 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:18.955750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:18.955750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:18.955750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:18.955750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:18.955750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:18.955750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:18.955750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:18.955750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:18.955750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:18.955750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:18.955750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:18.955750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:18.955750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:18.955750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:18.956629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:18.956629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:18.956629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:18.956629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:18.956629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:18.956629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:18.956629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:18.956629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:18.956629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:18.956629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:18.956629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:18.956629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:18.956629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:18.956629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:18.956629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:18.956629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:18.956629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:18.957520 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:18.957520 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:18.957520 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:18.957520 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:18.959672 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 328/655] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:18.960601 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816d634-675f-41ef-b34b-eea868a1f742 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:18.965179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-67cf9495-583f-469d-993b-5a7232f55736 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 04:09:18.965179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:18.965179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:18.965179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:18.965179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:18.965179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:18.965179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:18.965179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:18.965179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:18.965179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:18.965179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:18.965179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:18.965179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:09:18.965179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:09:18.965179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:09:18.965179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:09:18.966150 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:18.966150 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:18.966150 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:09:18.966150 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:18.966150 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:09:18.966150 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:18.966150 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:18.966150 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:18.966150 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:18.966150 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:18.966150 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:18.966150 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:18.966150 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:18.966150 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:18.966150 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:18.966150 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:18.966150 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:18.966150 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:18.967081 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:18.967081 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:18.967081 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:09:18.967081 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:09:18.967081 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:09:18.967081 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:09:18.967081 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:09:18.967081 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:09:18.967081 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 04:09:18.967081 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:18.967081 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 328/656] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 27 04:09:18 2022] GET /identity/v3/users/neutron => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:18.967081 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf9495-583f-469d-993b-5a7232f55736 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:19.080868 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf9495-583f-469d-993b-5a7232f55736 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:19.090836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816d634-675f-41ef-b34b-eea868a1f742 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:19.095241 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:19.118685 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a3811-1f2d-4872-9df6-545979c9d004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:19.119737 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a3811-1f2d-4872-9df6-545979c9d004 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:19.120276 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a3811-1f2d-4872-9df6-545979c9d004 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:19.120757 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a3811-1f2d-4872-9df6-545979c9d004 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:19.121673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a3811-1f2d-4872-9df6-545979c9d004 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:19.122275 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a3811-1f2d-4872-9df6-545979c9d004 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:19.123848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a3811-1f2d-4872-9df6-545979c9d004 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:19.131347 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551617b-fd9c-4605-a562-a793776d5cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:19.132216 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551617b-fd9c-4605-a562-a793776d5cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:19.132406 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551617b-fd9c-4605-a562-a793776d5cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:19.132639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551617b-fd9c-4605-a562-a793776d5cb4 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:19.133345 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5551617b-fd9c-4605-a562-a793776d5cb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:19.133650 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5551617b-fd9c-4605-a562-a793776d5cb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:19.135082 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5551617b-fd9c-4605-a562-a793776d5cb4 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:19.153528 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 329/657] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:19.153528 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551617b-fd9c-4605-a562-a793776d5cb4 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:19.171845 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:19.264124 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 329/658] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:19.265322 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a3811-1f2d-4872-9df6-545979c9d004 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:19.274351 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551617b-fd9c-4605-a562-a793776d5cb4 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:19.310469 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c522ee-b22e-4a7a-9743-bee7d00108f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:19.311075 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c522ee-b22e-4a7a-9743-bee7d00108f2 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:19.311400 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c522ee-b22e-4a7a-9743-bee7d00108f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:19.311704 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c522ee-b22e-4a7a-9743-bee7d00108f2 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:19.312488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c522ee-b22e-4a7a-9743-bee7d00108f2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:19.312793 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c522ee-b22e-4a7a-9743-bee7d00108f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:19.314665 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c522ee-b22e-4a7a-9743-bee7d00108f2 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:19.337793 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 330/659] 10.222.0.46 () {56 vars in 1139 bytes} [Wed Jul 27 04:09:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:19.337793 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8649556-513a-4ca5-b878-3f619fc74144 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:19.337793 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8649556-513a-4ca5-b878-3f619fc74144 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:19.337793 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8649556-513a-4ca5-b878-3f619fc74144 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:19.337793 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 331/660] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:19.337793 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8649556-513a-4ca5-b878-3f619fc74144 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:19.375603 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a3811-1f2d-4872-9df6-545979c9d004 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:19.411450 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a501a4-d1b7-4baf-971e-711f3ef3b43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:19.412172 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a501a4-d1b7-4baf-971e-711f3ef3b43b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:19.412507 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a501a4-d1b7-4baf-971e-711f3ef3b43b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:19.412788 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a501a4-d1b7-4baf-971e-711f3ef3b43b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:19.420235 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a501a4-d1b7-4baf-971e-711f3ef3b43b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:19.420400 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:19.421679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a501a4-d1b7-4baf-971e-711f3ef3b43b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:19.433729 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-74a501a4-d1b7-4baf-971e-711f3ef3b43b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 27 04:09:19.433729 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:19.433729 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:19.433729 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:19.433729 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:19.433729 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:19.433729 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:19.433729 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:19.433729 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:19.433729 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:19.433729 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:19.433729 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:19.433729 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:09:19.433729 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:09:19.433729 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:09:19.433729 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:09:19.434807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:19.434807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:19.434807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:09:19.434807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:19.434807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:09:19.434807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:19.434807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:19.434807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:19.434807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:19.434807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:19.434807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:19.434807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:19.434807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:19.434807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:19.434807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:19.434807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:19.434807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:19.434807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:19.435749 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:19.435749 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:19.435749 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:09:19.435749 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:09:19.435749 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:09:19.435749 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:09:19.435749 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:09:19.435749 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:09:19.435749 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 27 04:09:19.435749 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:19.435749 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 330/661] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:09:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:19.435749 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a501a4-d1b7-4baf-971e-711f3ef3b43b None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:19.461600 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8649556-513a-4ca5-b878-3f619fc74144 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:19.480613 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:19.497664 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c833ead-2db6-4f84-ad52-bd5635d5119f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:19.498342 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c833ead-2db6-4f84-ad52-bd5635d5119f None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:19.498619 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c833ead-2db6-4f84-ad52-bd5635d5119f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:19.498892 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c833ead-2db6-4f84-ad52-bd5635d5119f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:19.506623 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c833ead-2db6-4f84-ad52-bd5635d5119f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:19.508113 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c833ead-2db6-4f84-ad52-bd5635d5119f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:19.520474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-9c833ead-2db6-4f84-ad52-bd5635d5119f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:19.520474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:19.520474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:19.520474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:19.520474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:19.520474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:19.520474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:19.520474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:19.520474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:19.520474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:19.520474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:19.520474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:19.520474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:09:19.520474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:09:19.520474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:09:19.521502 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:09:19.521502 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:19.521502 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:19.521502 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:19.521502 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:19.521502 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:19.521502 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:19.521502 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:19.521502 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:19.521502 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:19.521502 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:19.521502 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:19.521502 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:19.521502 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:19.521502 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:19.521502 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:19.521502 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:19.522461 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:09:19.522461 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:09:19.522461 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:09:19.522461 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:09:19.522461 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:09:19.522461 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:09:19.522461 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:19.522461 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:19.522461 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 332/662] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:19.522971 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63aa708-c0e1-4674-b50c-1a633c06a01c None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:19.523041 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63aa708-c0e1-4674-b50c-1a633c06a01c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:19.523714 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63aa708-c0e1-4674-b50c-1a633c06a01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:19.542243 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-f63aa708-c0e1-4674-b50c-1a633c06a01c None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:19.543500 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f63aa708-c0e1-4674-b50c-1a633c06a01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:19.560342 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a501a4-d1b7-4baf-971e-711f3ef3b43b None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:19.596770 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ead743-35b4-40da-b01b-c39fc7cdf04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:19.597403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ead743-35b4-40da-b01b-c39fc7cdf04a None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:19.597684 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ead743-35b4-40da-b01b-c39fc7cdf04a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:19.597957 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ead743-35b4-40da-b01b-c39fc7cdf04a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:19.626363 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 333/663] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:09:19.626363 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63aa708-c0e1-4674-b50c-1a633c06a01c None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:19.682247 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ead743-35b4-40da-b01b-c39fc7cdf04a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:19.683913 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ead743-35b4-40da-b01b-c39fc7cdf04a None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:19.691631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:19.716793 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:19.738940 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63aa708-c0e1-4674-b50c-1a633c06a01c None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:19.776992 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c6157-1ee6-44ce-bf69-f0384f9c2359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:19.777614 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c6157-1ee6-44ce-bf69-f0384f9c2359 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:19.777892 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c6157-1ee6-44ce-bf69-f0384f9c2359 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:19.778189 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c6157-1ee6-44ce-bf69-f0384f9c2359 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:19.778988 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c6157-1ee6-44ce-bf69-f0384f9c2359 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:19.779292 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c6157-1ee6-44ce-bf69-f0384f9c2359 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:19.780721 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c6157-1ee6-44ce-bf69-f0384f9c2359 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:19.794411 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 331/664] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:09:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:09:19.795313 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ead743-35b4-40da-b01b-c39fc7cdf04a None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:19.799603 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 334/665] 10.222.0.46 () {56 vars in 1137 bytes} [Wed Jul 27 04:09:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:19.799878 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c6157-1ee6-44ce-bf69-f0384f9c2359 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:19.903377 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c6157-1ee6-44ce-bf69-f0384f9c2359 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:19.916072 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ead743-35b4-40da-b01b-c39fc7cdf04a None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:19.925995 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:19.944912 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197fd7a5-9229-4baa-a8f1-e0c266f21225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:19.945561 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fd7a5-9229-4baa-a8f1-e0c266f21225 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:19.945848 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fd7a5-9229-4baa-a8f1-e0c266f21225 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:19.946263 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fd7a5-9229-4baa-a8f1-e0c266f21225 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:19.954842 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:19.957645 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197fd7a5-9229-4baa-a8f1-e0c266f21225 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:09:19.961170 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5f508-774e-4024-85e3-80fe20b412e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:19.961815 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5f508-774e-4024-85e3-80fe20b412e9 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:19.962125 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5f508-774e-4024-85e3-80fe20b412e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:19.962396 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5f508-774e-4024-85e3-80fe20b412e9 None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:19.962602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-197fd7a5-9229-4baa-a8f1-e0c266f21225 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:19.962602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:19.962602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:19.962602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:19.962602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:19.962602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:19.962602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:19.962602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:19.962602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:19.962602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:19.962602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:19.962602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:19.962602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:09:19.962602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:09:19.962602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:09:19.962602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:09:19.963499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:09:19.963499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:09:19.963499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:09:19.963499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:09:19.963499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:19.963499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:19.963499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:19.963499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:19.963499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:19.963499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:19.963499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:19.963499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:19.963499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:19.963499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:19.963499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:19.963499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:19.964330 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:19.964330 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:19.964330 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:19.964330 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:19.964330 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:09:19.964330 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:09:19.964330 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:09:19.964330 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:09:19.964330 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:09:19.964330 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:09:19.964330 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:19.964330 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:19.964968 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb5f508-774e-4024-85e3-80fe20b412e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:19.964968 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb5f508-774e-4024-85e3-80fe20b412e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:19.964968 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb5f508-774e-4024-85e3-80fe20b412e9 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:19.968710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 335/666] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 27 04:09:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:19.969545 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197fd7a5-9229-4baa-a8f1-e0c266f21225 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:19.977450 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 332/667] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:09:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:19.978342 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5f508-774e-4024-85e3-80fe20b412e9 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:20.073739 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197fd7a5-9229-4baa-a8f1-e0c266f21225 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:20.079314 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5f508-774e-4024-85e3-80fe20b412e9 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:20.106814 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7b9af5-39cf-499e-ab6e-fb77db2e69cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:20.107461 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b9af5-39cf-499e-ab6e-fb77db2e69cc None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:20.107746 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b9af5-39cf-499e-ab6e-fb77db2e69cc None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:20.108021 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b9af5-39cf-499e-ab6e-fb77db2e69cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:20.113136 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a532225e-e9f4-4395-8b07-f806aa93330d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:20.113759 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532225e-e9f4-4395-8b07-f806aa93330d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:20.114067 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532225e-e9f4-4395-8b07-f806aa93330d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:20.114344 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532225e-e9f4-4395-8b07-f806aa93330d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:20.118774 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7b9af5-39cf-499e-ab6e-fb77db2e69cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:20.120232 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7b9af5-39cf-499e-ab6e-fb77db2e69cc None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:20.135402 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-ad7b9af5-39cf-499e-ab6e-fb77db2e69cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:20.135402 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:20.135402 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:20.135402 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:20.135402 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:20.135402 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:20.135402 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:20.135402 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:20.135402 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:20.135402 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:20.135402 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:20.135402 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:20.135402 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:20.135402 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:20.135402 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:20.136353 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:20.136353 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:20.136353 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:20.136353 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:20.136353 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:20.136353 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:20.136353 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:20.136353 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:20.136353 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:20.136353 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:20.136353 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:20.136353 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:20.136353 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:20.136353 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:20.136353 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:20.136353 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:20.136353 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:20.137792 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:20.137792 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:20.137792 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:20.137792 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:20.137792 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 336/668] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:20.137792 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7b9af5-39cf-499e-ab6e-fb77db2e69cc None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:20.172166 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:20.175599 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:20.186184 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a532225e-e9f4-4395-8b07-f806aa93330d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:20.187623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a532225e-e9f4-4395-8b07-f806aa93330d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:20.242846 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7b9af5-39cf-499e-ab6e-fb77db2e69cc None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:20.275291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6089734d-4ef5-437f-8193-61ff1554c22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:20.275906 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6089734d-4ef5-437f-8193-61ff1554c22b None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:20.276183 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6089734d-4ef5-437f-8193-61ff1554c22b None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:20.276452 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6089734d-4ef5-437f-8193-61ff1554c22b None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:20.277194 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6089734d-4ef5-437f-8193-61ff1554c22b None admin] RBAC: Adding query filter params (name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:20.277500 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6089734d-4ef5-437f-8193-61ff1554c22b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:20.278896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6089734d-4ef5-437f-8193-61ff1554c22b None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:20.284785 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 333/669] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:09:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:09:20.285631 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a532225e-e9f4-4395-8b07-f806aa93330d None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:20.291594 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 337/670] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 27 04:09:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:20.291594 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fadc17-21d3-4fb3-9a2a-f2ed9ad60015 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:20.291594 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fadc17-21d3-4fb3-9a2a-f2ed9ad60015 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:20.291594 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fadc17-21d3-4fb3-9a2a-f2ed9ad60015 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:20.291594 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 338/671] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:20.291594 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fadc17-21d3-4fb3-9a2a-f2ed9ad60015 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:20.379851 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:20.388970 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a532225e-e9f4-4395-8b07-f806aa93330d None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:20.391970 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:20.400238 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fadc17-21d3-4fb3-9a2a-f2ed9ad60015 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:20.423777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd541ae-19fb-4c74-9360-63804d0bac3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:20.424323 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd541ae-19fb-4c74-9360-63804d0bac3b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:20.424598 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd541ae-19fb-4c74-9360-63804d0bac3b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:20.424878 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd541ae-19fb-4c74-9360-63804d0bac3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:20.425659 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd541ae-19fb-4c74-9360-63804d0bac3b None admin] RBAC: Adding query filter params (role.id=6521132834a64b029695bbdd5eb251d2, scope.project.id=e646d320aa5a4f78bfccaf941cf46478, user.id=134ef452d7814c73b7c371a1cdbcbc00) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:20.425963 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd541ae-19fb-4c74-9360-63804d0bac3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:20.427431 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd541ae-19fb-4c74-9360-63804d0bac3b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:20.433062 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 334/672] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:09:20 2022] GET /identity/v3/role_assignments?user.id=134ef452d7814c73b7c371a1cdbcbc00&scope.project.id=e646d320aa5a4f78bfccaf941cf46478&role.id=6521132834a64b029695bbdd5eb251d2 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:20.433824 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd541ae-19fb-4c74-9360-63804d0bac3b None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:20.434369 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a515f-173b-4708-a739-aa8a615b7f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:20.434992 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a515f-173b-4708-a739-aa8a615b7f62 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:20.435276 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a515f-173b-4708-a739-aa8a615b7f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:20.435553 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a515f-173b-4708-a739-aa8a615b7f62 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:20.436254 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42a515f-173b-4708-a739-aa8a615b7f62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:20.436559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42a515f-173b-4708-a739-aa8a615b7f62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:20.437945 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42a515f-173b-4708-a739-aa8a615b7f62 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:20.456755 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 339/673] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:20.456755 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a515f-173b-4708-a739-aa8a615b7f62 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:20.567784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd541ae-19fb-4c74-9360-63804d0bac3b None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:20.573846 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a515f-173b-4708-a739-aa8a615b7f62 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:20.608114 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c23c3-2c38-4898-a2a8-9deb9837ad0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:20.608782 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c23c3-2c38-4898-a2a8-9deb9837ad0f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:20.609097 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c23c3-2c38-4898-a2a8-9deb9837ad0f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:20.609381 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c23c3-2c38-4898-a2a8-9deb9837ad0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:20.619364 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9024563-1633-4e6e-9841-6bd6a0c93f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:20.620250 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9024563-1633-4e6e-9841-6bd6a0c93f6e None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:20.620379 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9024563-1633-4e6e-9841-6bd6a0c93f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:20.620559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9024563-1633-4e6e-9841-6bd6a0c93f6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:20.621468 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2c23c3-2c38-4898-a2a8-9deb9837ad0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:20.622702 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2c23c3-2c38-4898-a2a8-9deb9837ad0f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:20.631426 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9024563-1633-4e6e-9841-6bd6a0c93f6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:20.632882 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9024563-1633-4e6e-9841-6bd6a0c93f6e None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:20.640537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-aa2c23c3-2c38-4898-a2a8-9deb9837ad0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:20.640537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:20.640537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:20.640537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:20.640537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:20.640537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:20.640537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:20.640537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:20.640537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:20.640537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:20.640537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:20.640537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:20.640537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:20.640537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:20.640537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:20.642057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:20.642057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:20.642057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:20.642057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:20.642057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:20.642057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:20.642057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:20.642057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:20.642057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:20.642057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:20.642057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:20.642057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:20.642057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:20.642057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:20.642057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:20.642057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:20.642057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:20.642949 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:20.642949 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:20.642949 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:20.642949 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:20.643956 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 335/674] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:20.644938 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04df8a3-17f8-4262-8680-3999f91b4760 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:20.645227 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04df8a3-17f8-4262-8680-3999f91b4760 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:20.645506 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04df8a3-17f8-4262-8680-3999f91b4760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:20.654022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-f9024563-1633-4e6e-9841-6bd6a0c93f6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:20.654022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:20.654022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:20.654022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:20.654022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:20.654022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:20.654022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:20.654022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:20.654022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:20.654022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:20.654022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:20.654022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:20.654022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:20.654022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:20.654022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:20.654896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:20.654896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:20.654896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:20.654896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:20.654896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:20.654896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:20.654896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:20.654896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:20.654896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:20.654896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:20.654896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:20.654896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:20.654896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:20.654896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:20.654896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:20.654896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:20.654896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:20.655779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:20.655779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:20.655779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:20.655779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:20.655779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 340/675] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:20.656459 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9024563-1633-4e6e-9841-6bd6a0c93f6e None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:20.663753 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-e04df8a3-17f8-4262-8680-3999f91b4760 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:20.665318 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e04df8a3-17f8-4262-8680-3999f91b4760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:20.667883 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:20.669210 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:20.740024 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 336/676] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:09:20.740790 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04df8a3-17f8-4262-8680-3999f91b4760 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:20.776111 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9024563-1633-4e6e-9841-6bd6a0c93f6e None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:20.812439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f86718-6f04-4ac2-804b-1bc8ffd1d64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:20.813817 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f86718-6f04-4ac2-804b-1bc8ffd1d64e None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:20.814334 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f86718-6f04-4ac2-804b-1bc8ffd1d64e None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:20.814827 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f86718-6f04-4ac2-804b-1bc8ffd1d64e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:20.821994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f86718-6f04-4ac2-804b-1bc8ffd1d64e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:20.823665 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f86718-6f04-4ac2-804b-1bc8ffd1d64e None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:20.835897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-c0f86718-6f04-4ac2-804b-1bc8ffd1d64e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:20.835897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:20.835897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:20.835897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:20.835897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:20.835897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:20.835897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:20.835897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:20.835897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:20.835897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:20.835897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:20.835897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:20.835897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:09:20.835897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:09:20.835897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:09:20.836852 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:09:20.836852 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:20.836852 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:20.836852 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:20.836852 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:20.836852 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:20.836852 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:20.836852 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:20.836852 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:20.836852 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:20.836852 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:20.836852 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:20.836852 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:20.836852 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:20.836852 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:20.836852 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:20.836852 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:20.837713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:09:20.837713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:09:20.837713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:09:20.837713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:09:20.837713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:09:20.837713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:09:20.837713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:20.837713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:20.843978 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 341/677] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:20.845462 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f86718-6f04-4ac2-804b-1bc8ffd1d64e None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:20.855492 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04df8a3-17f8-4262-8680-3999f91b4760 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:20.893192 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205fc0d5-d18a-42d0-9450-38c60001327d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:20.894375 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205fc0d5-d18a-42d0-9450-38c60001327d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:20.894657 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205fc0d5-d18a-42d0-9450-38c60001327d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:20.894930 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205fc0d5-d18a-42d0-9450-38c60001327d None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:20.895979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205fc0d5-d18a-42d0-9450-38c60001327d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:20.896283 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205fc0d5-d18a-42d0-9450-38c60001327d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:20.897953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205fc0d5-d18a-42d0-9450-38c60001327d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:20.904566 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:20.911062 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 337/678] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:20.911957 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205fc0d5-d18a-42d0-9450-38c60001327d None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:20.925763 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:20.969385 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f86718-6f04-4ac2-804b-1bc8ffd1d64e None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:21.005812 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec70e5-d626-4572-995c-81a6ecd1e491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:21.006385 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec70e5-d626-4572-995c-81a6ecd1e491 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:21.006658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec70e5-d626-4572-995c-81a6ecd1e491 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:21.006924 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec70e5-d626-4572-995c-81a6ecd1e491 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:21.007618 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ec70e5-d626-4572-995c-81a6ecd1e491 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:21.008017 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ec70e5-d626-4572-995c-81a6ecd1e491 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:21.009288 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ec70e5-d626-4572-995c-81a6ecd1e491 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:21.019787 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205fc0d5-d18a-42d0-9450-38c60001327d None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:21.022714 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 342/679] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:21.023060 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec70e5-d626-4572-995c-81a6ecd1e491 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:21.059119 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23615dd3-7e5b-49b8-ae51-273fedb3e1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:21.059762 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23615dd3-7e5b-49b8-ae51-273fedb3e1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:21.060039 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23615dd3-7e5b-49b8-ae51-273fedb3e1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:21.060308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23615dd3-7e5b-49b8-ae51-273fedb3e1e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:21.071126 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23615dd3-7e5b-49b8-ae51-273fedb3e1e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:09:21.076386 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-23615dd3-7e5b-49b8-ae51-273fedb3e1e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:09:21.076386 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:21.076386 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:21.076386 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:21.076386 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:21.076386 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:21.076386 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:21.076386 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:21.076386 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:21.076386 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:21.076386 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:21.076386 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:21.076386 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:09:21.076386 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:09:21.076386 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:09:21.076386 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:09:21.077304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:09:21.077304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:09:21.077304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:09:21.077304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:09:21.077304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:21.077304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:21.077304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:21.077304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:21.077304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:21.077304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:21.077304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:21.077304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:21.077304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:21.077304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:21.077304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:21.077304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:21.078706 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:21.078706 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:21.078706 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:21.078706 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:21.078706 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:09:21.078706 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:09:21.078706 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:09:21.078706 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:09:21.078706 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:09:21.078706 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:09:21.078706 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:09:21.078706 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:21.078706 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 338/680] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:09:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:21.078706 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23615dd3-7e5b-49b8-ae51-273fedb3e1e4 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:21.128826 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec70e5-d626-4572-995c-81a6ecd1e491 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:21.145697 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:21.155587 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:21.164372 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fce68-ba80-466f-951c-ed240ef554a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:21.165005 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fce68-ba80-466f-951c-ed240ef554a6 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:21.165337 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fce68-ba80-466f-951c-ed240ef554a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:21.165619 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fce68-ba80-466f-951c-ed240ef554a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:21.166402 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8fce68-ba80-466f-951c-ed240ef554a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:21.166708 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8fce68-ba80-466f-951c-ed240ef554a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:21.168194 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8fce68-ba80-466f-951c-ed240ef554a6 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:21.180093 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 343/681] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:09:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:21.185478 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fce68-ba80-466f-951c-ed240ef554a6 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:21.194890 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23615dd3-7e5b-49b8-ae51-273fedb3e1e4 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:21.232310 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3226052b-54ba-4325-a839-13de5404589d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:21.233050 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226052b-54ba-4325-a839-13de5404589d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:21.233335 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226052b-54ba-4325-a839-13de5404589d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:21.233620 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226052b-54ba-4325-a839-13de5404589d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:21.241715 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3226052b-54ba-4325-a839-13de5404589d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:21.243171 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3226052b-54ba-4325-a839-13de5404589d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:21.255390 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-3226052b-54ba-4325-a839-13de5404589d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:21.255390 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:21.255390 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:21.255390 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:21.255390 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:21.255390 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:21.255390 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:21.255390 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:21.255390 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:21.255390 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:21.255390 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:21.255390 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:21.255390 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:09:21.255390 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:09:21.255390 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:09:21.256403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:09:21.256403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:21.256403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:21.256403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:21.256403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:21.256403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:21.256403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:21.256403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:21.256403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:21.256403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:21.256403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:21.256403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:21.256403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:21.256403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:21.256403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:21.256403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:21.256403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:21.257337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:09:21.257337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:09:21.257337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:09:21.257337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:09:21.257337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:09:21.257337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:09:21.257337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:21.257337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:21.257337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 339/682] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:21.257830 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3226052b-54ba-4325-a839-13de5404589d None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:21.300062 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fce68-ba80-466f-951c-ed240ef554a6 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:21.335636 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e312c3-e04f-4438-ad40-7abc1cb6a244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:21.336633 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e312c3-e04f-4438-ad40-7abc1cb6a244 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:21.337113 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e312c3-e04f-4438-ad40-7abc1cb6a244 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:21.337565 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e312c3-e04f-4438-ad40-7abc1cb6a244 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:21.346452 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e312c3-e04f-4438-ad40-7abc1cb6a244 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:21.348102 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e312c3-e04f-4438-ad40-7abc1cb6a244 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:21.360448 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-98e312c3-e04f-4438-ad40-7abc1cb6a244 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 04:09:21.360448 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:21.360448 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:21.360448 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:21.360448 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:21.360448 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:21.360448 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:21.360448 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:21.360448 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:21.360448 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:21.360448 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:21.360448 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:21.360448 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:09:21.360448 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:09:21.360448 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:09:21.360448 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:09:21.361366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:21.361366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:21.361366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:09:21.361366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:21.361366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:09:21.361366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:21.361366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:21.361366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:21.361366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:21.361366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:21.361366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:21.361366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:21.361366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:21.361366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:21.361366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:21.361366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:21.361366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:21.361366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:21.362407 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:21.362407 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:21.362407 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:09:21.362407 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:09:21.362407 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:09:21.362407 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:09:21.362407 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:09:21.362407 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:09:21.362407 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 04:09:21.362407 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:21.362967 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3226052b-54ba-4325-a839-13de5404589d None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:21.363753 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 344/683] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:09:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:21.364567 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e312c3-e04f-4438-ad40-7abc1cb6a244 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:21.379560 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:21.398479 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc67d3a-cee7-44c0-9deb-a1757711e876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:21.399107 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc67d3a-cee7-44c0-9deb-a1757711e876 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:21.399424 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc67d3a-cee7-44c0-9deb-a1757711e876 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:21.399670 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc67d3a-cee7-44c0-9deb-a1757711e876 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:21.400392 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc67d3a-cee7-44c0-9deb-a1757711e876 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:21.400700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc67d3a-cee7-44c0-9deb-a1757711e876 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:21.402088 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc67d3a-cee7-44c0-9deb-a1757711e876 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:21.410506 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 340/684] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:09:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:21.411350 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc67d3a-cee7-44c0-9deb-a1757711e876 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:21.420959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:21.484547 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e312c3-e04f-4438-ad40-7abc1cb6a244 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:21.520667 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc67d3a-cee7-44c0-9deb-a1757711e876 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:21.523498 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8dcef-fc90-434b-9cda-0aba4924fc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:21.523811 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8dcef-fc90-434b-9cda-0aba4924fc03 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:21.524097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8dcef-fc90-434b-9cda-0aba4924fc03 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:21.524367 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8dcef-fc90-434b-9cda-0aba4924fc03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:21.525157 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8dcef-fc90-434b-9cda-0aba4924fc03 None admin] RBAC: Adding query filter params (role.id=6521132834a64b029695bbdd5eb251d2, scope.project.id=e646d320aa5a4f78bfccaf941cf46478, user.id=b4847f3736d844d78a206b7eab5f5fcd) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:21.525464 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8dcef-fc90-434b-9cda-0aba4924fc03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:21.526924 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8dcef-fc90-434b-9cda-0aba4924fc03 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:21.538021 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 345/685] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:09:21 2022] GET /identity/v3/role_assignments?user.id=b4847f3736d844d78a206b7eab5f5fcd&scope.project.id=e646d320aa5a4f78bfccaf941cf46478&role.id=6521132834a64b029695bbdd5eb251d2 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:21.538021 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8dcef-fc90-434b-9cda-0aba4924fc03 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:21.566784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76523432-15a2-447f-8b0b-fadbdfe45882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:21.567525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76523432-15a2-447f-8b0b-fadbdfe45882 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:21.567930 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76523432-15a2-447f-8b0b-fadbdfe45882 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:21.568203 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76523432-15a2-447f-8b0b-fadbdfe45882 None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:21.569053 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76523432-15a2-447f-8b0b-fadbdfe45882 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:21.569361 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76523432-15a2-447f-8b0b-fadbdfe45882 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:21.570902 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76523432-15a2-447f-8b0b-fadbdfe45882 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:21.592261 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 341/686] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:09:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:21.593276 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76523432-15a2-447f-8b0b-fadbdfe45882 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:21.649948 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:21.686648 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8dcef-fc90-434b-9cda-0aba4924fc03 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:21.703039 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:21.719901 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76523432-15a2-447f-8b0b-fadbdfe45882 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:21.738910 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a370f0-9f49-440d-bb5f-58d03aa4194c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:21.739541 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a370f0-9f49-440d-bb5f-58d03aa4194c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:21.739824 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a370f0-9f49-440d-bb5f-58d03aa4194c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:21.740098 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a370f0-9f49-440d-bb5f-58d03aa4194c None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:21.740904 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a370f0-9f49-440d-bb5f-58d03aa4194c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:21.741214 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a370f0-9f49-440d-bb5f-58d03aa4194c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:21.742771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a370f0-9f49-440d-bb5f-58d03aa4194c None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:21.763385 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 346/687] 10.222.0.46 () {56 vars in 1135 bytes} [Wed Jul 27 04:09:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:21.764071 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a370f0-9f49-440d-bb5f-58d03aa4194c None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:21.767512 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac97616-2006-4b9e-bd38-51a14f6c5a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:21.768278 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac97616-2006-4b9e-bd38-51a14f6c5a3d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:21.768686 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac97616-2006-4b9e-bd38-51a14f6c5a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:21.769108 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac97616-2006-4b9e-bd38-51a14f6c5a3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:21.780664 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac97616-2006-4b9e-bd38-51a14f6c5a3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:21.785993 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac97616-2006-4b9e-bd38-51a14f6c5a3d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:21.808393 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-1ac97616-2006-4b9e-bd38-51a14f6c5a3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:21.808393 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:21.808393 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:21.808393 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:21.808393 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:21.808393 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:21.808393 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:21.808393 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:21.808393 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:21.808393 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:21.808393 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:21.808393 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:21.808393 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:21.808393 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:21.808393 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:21.809306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:21.809306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:21.809306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:21.809306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:21.809306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:21.809306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:21.809306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:21.809306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:21.809306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:21.809306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:21.809306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:21.809306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:21.809306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:21.809306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:21.809306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:21.809306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:21.809306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:21.810237 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:21.810237 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:21.810237 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:21.810237 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:21.816041 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 342/688] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:21.817219 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac97616-2006-4b9e-bd38-51a14f6c5a3d None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:21.882151 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a370f0-9f49-440d-bb5f-58d03aa4194c None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:21.919273 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e940c-8337-4ad0-a88a-3356c8156017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:21.919826 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e940c-8337-4ad0-a88a-3356c8156017 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:21.920104 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e940c-8337-4ad0-a88a-3356c8156017 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:21.920372 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e940c-8337-4ad0-a88a-3356c8156017 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:21.921230 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e940c-8337-4ad0-a88a-3356c8156017 None admin] RBAC: Adding query filter params (role.id=6521132834a64b029695bbdd5eb251d2, scope.project.id=e646d320aa5a4f78bfccaf941cf46478, user.id=3e98a41a0f304fc2bb36a18a77a2d729) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:21.921596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e940c-8337-4ad0-a88a-3356c8156017 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:21.923153 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e940c-8337-4ad0-a88a-3356c8156017 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:21.934354 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 347/689] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:09:21 2022] GET /identity/v3/role_assignments?user.id=3e98a41a0f304fc2bb36a18a77a2d729&scope.project.id=e646d320aa5a4f78bfccaf941cf46478&role.id=6521132834a64b029695bbdd5eb251d2 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:21.935291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e940c-8337-4ad0-a88a-3356c8156017 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:21.941950 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:21.953180 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac97616-2006-4b9e-bd38-51a14f6c5a3d None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:21.998161 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27298948-69db-482d-bcb5-6e064d63145d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:21.998953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27298948-69db-482d-bcb5-6e064d63145d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:21.999250 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27298948-69db-482d-bcb5-6e064d63145d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:21.999530 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27298948-69db-482d-bcb5-6e064d63145d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:22.009410 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:22.090843 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27298948-69db-482d-bcb5-6e064d63145d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:22.092343 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27298948-69db-482d-bcb5-6e064d63145d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:22.195305 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e940c-8337-4ad0-a88a-3356c8156017 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:22.207316 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:22.222998 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 343/690] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:09:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:09:22.227683 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27298948-69db-482d-bcb5-6e064d63145d None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:22.250640 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba3296-4ae3-4abf-bfc8-e21bc1f01085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:22.252066 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba3296-4ae3-4abf-bfc8-e21bc1f01085 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:22.252356 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba3296-4ae3-4abf-bfc8-e21bc1f01085 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:22.252632 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba3296-4ae3-4abf-bfc8-e21bc1f01085 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:22.253350 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ba3296-4ae3-4abf-bfc8-e21bc1f01085 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:22.253659 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ba3296-4ae3-4abf-bfc8-e21bc1f01085 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:22.255658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ba3296-4ae3-4abf-bfc8-e21bc1f01085 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:22.274562 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 348/691] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:22.285467 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba3296-4ae3-4abf-bfc8-e21bc1f01085 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:22.362830 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27298948-69db-482d-bcb5-6e064d63145d None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:22.398376 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:22.401660 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba3296-4ae3-4abf-bfc8-e21bc1f01085 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:22.411824 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78144e-a2fd-4763-a3a8-41a0dbf648c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:22.412653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78144e-a2fd-4763-a3a8-41a0dbf648c8 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:22.412990 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78144e-a2fd-4763-a3a8-41a0dbf648c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:22.413266 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78144e-a2fd-4763-a3a8-41a0dbf648c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:22.425591 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba78144e-a2fd-4763-a3a8-41a0dbf648c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:22.427089 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba78144e-a2fd-4763-a3a8-41a0dbf648c8 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:22.444434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-ba78144e-a2fd-4763-a3a8-41a0dbf648c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:22.444434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:22.444434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:22.444434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:22.444434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:22.444434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:22.444434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:22.444434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:22.444434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:22.444434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:22.444434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:22.444434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:22.444434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:22.444434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:22.444434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:22.445346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:22.445346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:22.445346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:22.445346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:22.445346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:22.445346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:22.445346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:22.445346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:22.445346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:22.445346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:22.445346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:22.445346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:22.445346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:22.445346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:22.445346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:22.445346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:22.445346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:22.446277 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:22.446277 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:22.446277 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:22.446277 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:22.446986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 344/692] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:09:22.447155 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a01fdef-83d4-48da-ad85-4711ca066f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:22.447795 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a01fdef-83d4-48da-ad85-4711ca066f98 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:22.448184 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a01fdef-83d4-48da-ad85-4711ca066f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:22.448471 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a01fdef-83d4-48da-ad85-4711ca066f98 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:22.453252 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78144e-a2fd-4763-a3a8-41a0dbf648c8 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:22.471205 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a01fdef-83d4-48da-ad85-4711ca066f98 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:22.472892 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a01fdef-83d4-48da-ad85-4711ca066f98 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:22.490776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-3a01fdef-83d4-48da-ad85-4711ca066f98 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 04:09:22.490776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:22.490776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:22.490776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:22.490776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:22.490776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:22.490776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:22.490776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:22.490776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:22.490776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:22.490776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:22.490776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:22.490776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:09:22.490776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:09:22.490776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:09:22.490776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:09:22.491836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:22.491836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:22.491836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:09:22.491836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:22.491836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:09:22.491836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:22.491836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:22.491836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:22.491836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:22.491836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:22.491836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:22.491836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:22.491836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:22.491836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:22.491836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:22.491836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:22.491836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:22.491836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:22.492752 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:22.492752 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:22.492752 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:09:22.492752 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:09:22.492752 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:09:22.492752 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:09:22.492752 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:09:22.492752 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:09:22.492752 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 04:09:22.492752 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:22.495404 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 349/693] 10.222.0.46 () {56 vars in 1093 bytes} [Wed Jul 27 04:09:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:22.501966 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a01fdef-83d4-48da-ad85-4711ca066f98 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:22.519529 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:22.593601 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78144e-a2fd-4763-a3a8-41a0dbf648c8 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:22.617862 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a01fdef-83d4-48da-ad85-4711ca066f98 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:22.640760 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f3299e-20d3-4ee5-b108-204a17724bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:22.641458 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3299e-20d3-4ee5-b108-204a17724bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:22.641747 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3299e-20d3-4ee5-b108-204a17724bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:22.642482 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3299e-20d3-4ee5-b108-204a17724bf1 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:22.643204 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f3299e-20d3-4ee5-b108-204a17724bf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:22.643790 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f3299e-20d3-4ee5-b108-204a17724bf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:22.645355 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f3299e-20d3-4ee5-b108-204a17724bf1 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:22.656068 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb895d-9963-4f2f-84bd-d42630d4dc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:22.656835 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb895d-9963-4f2f-84bd-d42630d4dc0d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:22.657859 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb895d-9963-4f2f-84bd-d42630d4dc0d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:22.658260 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb895d-9963-4f2f-84bd-d42630d4dc0d None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:22.659157 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb895d-9963-4f2f-84bd-d42630d4dc0d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:22.659563 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb895d-9963-4f2f-84bd-d42630d4dc0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:22.661080 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb895d-9963-4f2f-84bd-d42630d4dc0d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:22.661232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 345/694] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:22.662638 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8e86a-a463-4c8b-aee8-0251c76b94b5 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:22.662933 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8e86a-a463-4c8b-aee8-0251c76b94b5 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:22.663264 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8e86a-a463-4c8b-aee8-0251c76b94b5 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:22.664748 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 346/695] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:22.666515 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8e86a-a463-4c8b-aee8-0251c76b94b5 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:22.690123 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 350/696] 10.222.0.46 () {56 vars in 1133 bytes} [Wed Jul 27 04:09:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:09:22.691497 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5d494-bd5a-4652-a5bf-021e7d52df59 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:22.691797 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5d494-bd5a-4652-a5bf-021e7d52df59 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:22.692073 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5d494-bd5a-4652-a5bf-021e7d52df59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:22.715849 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-63e5d494-bd5a-4652-a5bf-021e7d52df59 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:22.718368 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-63e5d494-bd5a-4652-a5bf-021e7d52df59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:22.731626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:22.793764 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8e86a-a463-4c8b-aee8-0251c76b94b5 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:22.797049 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:22.820248 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 351/697] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:09:22.821478 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5d494-bd5a-4652-a5bf-021e7d52df59 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:22.833575 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb7565d-0543-4ae2-aab1-52e6fa8e3b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:22.834399 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb7565d-0543-4ae2-aab1-52e6fa8e3b5a None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:22.834859 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb7565d-0543-4ae2-aab1-52e6fa8e3b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:22.835257 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb7565d-0543-4ae2-aab1-52e6fa8e3b5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:22.842719 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb7565d-0543-4ae2-aab1-52e6fa8e3b5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:22.844507 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb7565d-0543-4ae2-aab1-52e6fa8e3b5a None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:22.857548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-abb7565d-0543-4ae2-aab1-52e6fa8e3b5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:22.857548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:22.857548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:22.857548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:22.857548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:22.857548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:22.857548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:22.857548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:22.857548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:22.857548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:22.857548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:22.857548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:22.857548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:09:22.857548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:09:22.857548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:09:22.858677 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:09:22.858677 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:22.858677 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:22.858677 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:22.858677 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:22.858677 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:22.858677 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:22.858677 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:22.858677 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:22.858677 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:22.858677 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:22.858677 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:22.858677 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:22.858677 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:22.858677 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:22.858677 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:22.858677 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:22.860525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:09:22.860525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:09:22.860525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:09:22.860525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:09:22.860525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:09:22.860525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:09:22.860525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:22.860525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:22.870069 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 347/698] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:09:22.870069 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb7565d-0543-4ae2-aab1-52e6fa8e3b5a None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:22.946266 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5d494-bd5a-4652-a5bf-021e7d52df59 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:22.986352 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b174e-d4a6-4748-b4cb-bd114bebebcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:22.987193 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b174e-d4a6-4748-b4cb-bd114bebebcc None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:22.987576 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b174e-d4a6-4748-b4cb-bd114bebebcc None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:22.987951 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b174e-d4a6-4748-b4cb-bd114bebebcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:22.989203 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb7565d-0543-4ae2-aab1-52e6fa8e3b5a None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:22.999672 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:23.032661 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252f05a-abcb-44c9-92c7-71698b53a999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:23.033571 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252f05a-abcb-44c9-92c7-71698b53a999 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:23.033959 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252f05a-abcb-44c9-92c7-71698b53a999 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:23.034414 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252f05a-abcb-44c9-92c7-71698b53a999 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:23.035205 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7252f05a-abcb-44c9-92c7-71698b53a999 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:23.036816 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7252f05a-abcb-44c9-92c7-71698b53a999 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:23.048855 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-7252f05a-abcb-44c9-92c7-71698b53a999 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 04:09:23.048855 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:23.048855 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 04:09:23.048855 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 04:09:23.048855 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 04:09:23.048855 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 04:09:23.048855 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 04:09:23.048855 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:23.048855 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 04:09:23.048855 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:23.048855 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 04:09:23.048855 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:23.048855 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:23.048855 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:23.048855 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:23.048855 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:23.050167 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:23.050167 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:23.050167 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:23.050167 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:23.050167 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:23.050167 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:23.050167 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:23.050167 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 04:09:23.050167 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 04:09:23.050167 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 04:09:23.050167 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 04:09:23.050167 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:23.050167 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:23.050167 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:23.050167 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:23.050167 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:23.051238 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:23.051238 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:23.051238 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:23.051238 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:23.051238 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:23.051238 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:23.051238 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:23.051238 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:23.051238 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:23.051238 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:23.051238 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:23.051238 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 04:09:23.051238 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:23.051238 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 04:09:23.051238 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:23.053335 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 348/699] 10.222.0.46 () {56 vars in 1129 bytes} [Wed Jul 27 04:09:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:23.054274 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252f05a-abcb-44c9-92c7-71698b53a999 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:23.079116 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6b174e-d4a6-4748-b4cb-bd114bebebcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:23.080830 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6b174e-d4a6-4748-b4cb-bd114bebebcc None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:23.095575 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:23.190790 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252f05a-abcb-44c9-92c7-71698b53a999 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:23.198785 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 352/700] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:09:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 04:09:23.199918 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b174e-d4a6-4748-b4cb-bd114bebebcc None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:23.227882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2491a-3f62-406c-8aa1-a04dd7563804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:23.228619 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2491a-3f62-406c-8aa1-a04dd7563804 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:23.228966 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2491a-3f62-406c-8aa1-a04dd7563804 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:23.229328 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2491a-3f62-406c-8aa1-a04dd7563804 None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:23.230356 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e2491a-3f62-406c-8aa1-a04dd7563804 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:23.230490 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e2491a-3f62-406c-8aa1-a04dd7563804 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:23.232015 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e2491a-3f62-406c-8aa1-a04dd7563804 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:23.245170 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 349/701] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:09:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:23.246186 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2491a-3f62-406c-8aa1-a04dd7563804 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:23.257058 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:23.328423 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b174e-d4a6-4748-b4cb-bd114bebebcc None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:23.359210 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:23.367443 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c14f95-32e6-4ba4-afe8-2d441e50c38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:23.368306 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c14f95-32e6-4ba4-afe8-2d441e50c38d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:23.368946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c14f95-32e6-4ba4-afe8-2d441e50c38d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:23.369163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c14f95-32e6-4ba4-afe8-2d441e50c38d None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:23.370011 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c14f95-32e6-4ba4-afe8-2d441e50c38d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:23.370427 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c14f95-32e6-4ba4-afe8-2d441e50c38d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:23.372061 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c14f95-32e6-4ba4-afe8-2d441e50c38d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:23.377219 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2491a-3f62-406c-8aa1-a04dd7563804 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:23.377949 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 353/702] 10.222.0.46 () {56 vars in 1124 bytes} [Wed Jul 27 04:09:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:23.378890 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c14f95-32e6-4ba4-afe8-2d441e50c38d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:23.423526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00343a6a-d7ed-4265-a823-d05e93ea05fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:23.424472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00343a6a-d7ed-4265-a823-d05e93ea05fc None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:23.425016 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00343a6a-d7ed-4265-a823-d05e93ea05fc None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:23.425495 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00343a6a-d7ed-4265-a823-d05e93ea05fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:23.441129 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00343a6a-d7ed-4265-a823-d05e93ea05fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:23.443668 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00343a6a-d7ed-4265-a823-d05e93ea05fc None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:23.464498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-00343a6a-d7ed-4265-a823-d05e93ea05fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:23.464498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:23.464498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:23.464498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:23.464498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:23.464498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:23.464498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:23.464498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:23.464498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:23.464498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:23.464498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:23.464498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:23.464498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:23.464498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:23.464498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:23.465622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:23.465622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:23.465622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:23.465622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:23.465622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:23.465622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:23.465622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:23.465622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:23.465622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:23.465622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:23.465622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:23.465622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:23.465622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:23.465622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:23.465622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:23.465622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:23.465622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:23.466664 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:23.466664 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:23.466664 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:23.466664 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:23.469937 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 350/703] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:23.471510 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00343a6a-d7ed-4265-a823-d05e93ea05fc None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:23.502925 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c14f95-32e6-4ba4-afe8-2d441e50c38d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:23.506285 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:23.549850 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ed74c-55e3-49bc-a034-d74a53deecb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:23.550756 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ed74c-55e3-49bc-a034-d74a53deecb2 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:23.551224 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ed74c-55e3-49bc-a034-d74a53deecb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:23.551661 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ed74c-55e3-49bc-a034-d74a53deecb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:23.553188 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ed74c-55e3-49bc-a034-d74a53deecb2 None admin] RBAC: Adding query filter params (role.id=6521132834a64b029695bbdd5eb251d2, scope.project.id=e646d320aa5a4f78bfccaf941cf46478, user.id=57b5fa7517034a39930ce7c9d12e1875) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:23.553684 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ed74c-55e3-49bc-a034-d74a53deecb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:23.555866 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ed74c-55e3-49bc-a034-d74a53deecb2 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:23.565592 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 354/704] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:09:23 2022] GET /identity/v3/role_assignments?user.id=57b5fa7517034a39930ce7c9d12e1875&scope.project.id=e646d320aa5a4f78bfccaf941cf46478&role.id=6521132834a64b029695bbdd5eb251d2 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:23.566845 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ed74c-55e3-49bc-a034-d74a53deecb2 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:23.583022 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00343a6a-d7ed-4265-a823-d05e93ea05fc None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:23.625233 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d5378e-e1bd-40eb-9ef3-dea2e798feae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:23.626888 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5378e-e1bd-40eb-9ef3-dea2e798feae None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:23.627292 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5378e-e1bd-40eb-9ef3-dea2e798feae None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:23.627835 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5378e-e1bd-40eb-9ef3-dea2e798feae None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:23.628643 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d5378e-e1bd-40eb-9ef3-dea2e798feae None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:23.629056 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d5378e-e1bd-40eb-9ef3-dea2e798feae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:23.630598 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d5378e-e1bd-40eb-9ef3-dea2e798feae None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:23.637352 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 351/705] 10.222.0.46 () {56 vars in 1124 bytes} [Wed Jul 27 04:09:23 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:23.740526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d57207-147c-41df-8a2f-2ac4b05a1587 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:23.740839 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d57207-147c-41df-8a2f-2ac4b05a1587 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:23.741004 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d57207-147c-41df-8a2f-2ac4b05a1587 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:23.742658 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 352/706] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:23.748469 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c75aef-c3b7-458d-a749-7cbf657ddff4 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:23.749085 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c75aef-c3b7-458d-a749-7cbf657ddff4 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:23.749566 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c75aef-c3b7-458d-a749-7cbf657ddff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:23.751556 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ed74c-55e3-49bc-a034-d74a53deecb2 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:23.766009 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:23.778878 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-78c75aef-c3b7-458d-a749-7cbf657ddff4 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:23.780161 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-78c75aef-c3b7-458d-a749-7cbf657ddff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:23.815060 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d710c21-c5c3-4fc2-8530-bb08a3662411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:23.816365 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d710c21-c5c3-4fc2-8530-bb08a3662411 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:23.816773 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d710c21-c5c3-4fc2-8530-bb08a3662411 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:23.817057 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d710c21-c5c3-4fc2-8530-bb08a3662411 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:23.818066 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d710c21-c5c3-4fc2-8530-bb08a3662411 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:23.818383 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d710c21-c5c3-4fc2-8530-bb08a3662411 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:23.820696 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d710c21-c5c3-4fc2-8530-bb08a3662411 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:23.839991 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 355/707] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:23.846201 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d710c21-c5c3-4fc2-8530-bb08a3662411 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:23.868992 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 353/708] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 04:09:23.883021 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:23.891057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c75aef-c3b7-458d-a749-7cbf657ddff4 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:23.985139 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d710c21-c5c3-4fc2-8530-bb08a3662411 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:24.022505 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c75aef-c3b7-458d-a749-7cbf657ddff4 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:24.028227 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030704a5-0af2-4238-9a2b-7ee0373ac8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:24.028884 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030704a5-0af2-4238-9a2b-7ee0373ac8f3 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:24.029164 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030704a5-0af2-4238-9a2b-7ee0373ac8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:24.029437 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030704a5-0af2-4238-9a2b-7ee0373ac8f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:24.037825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030704a5-0af2-4238-9a2b-7ee0373ac8f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:24.039338 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030704a5-0af2-4238-9a2b-7ee0373ac8f3 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:24.064692 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864da125-4123-46d5-806c-5b0aafac4e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:24.065413 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864da125-4123-46d5-806c-5b0aafac4e77 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:24.065698 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864da125-4123-46d5-806c-5b0aafac4e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:24.065993 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864da125-4123-46d5-806c-5b0aafac4e77 None admin] PATH_INFO: `/v3/services/network` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:24.066708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864da125-4123-46d5-806c-5b0aafac4e77 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:24.068160 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864da125-4123-46d5-806c-5b0aafac4e77 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:24.070456 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:24.080784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-864da125-4123-46d5-806c-5b0aafac4e77 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 04:09:24.080784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:24.080784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 04:09:24.080784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 04:09:24.080784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 04:09:24.080784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 04:09:24.080784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 04:09:24.080784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:24.080784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 04:09:24.080784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:24.080784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 04:09:24.080784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:24.080784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:24.080784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:24.080784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:24.080784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:24.080784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:24.082943 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:24.082943 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:24.082943 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:24.082943 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:24.082943 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:24.082943 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:24.082943 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 04:09:24.082943 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 04:09:24.082943 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 04:09:24.082943 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 04:09:24.082943 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:24.082943 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:24.082943 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:24.082943 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:24.082943 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:24.082943 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:24.083980 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:24.083980 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:24.083980 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:24.083980 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:24.083980 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:24.083980 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:24.083980 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:24.083980 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:24.083980 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:24.083980 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:24.083980 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 04:09:24.083980 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:24.083980 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 04:09:24.083980 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:24.083980 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 354/709] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:23 2022] GET /identity/v3/services/network => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:24.091036 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864da125-4123-46d5-806c-5b0aafac4e77 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:24.174340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-030704a5-0af2-4238-9a2b-7ee0373ac8f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:24.174340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:24.174340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:24.174340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:24.174340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:24.174340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:24.174340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:24.174340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:24.174340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:24.174340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:24.174340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:24.174340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:24.174340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:09:24.174340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:09:24.174340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:09:24.176523 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:09:24.176523 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:24.176523 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:24.176523 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:24.176523 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:24.176523 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:24.176523 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:24.176523 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:24.176523 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:24.176523 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:24.176523 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:24.176523 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:24.176523 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:24.176523 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:24.176523 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:24.176523 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:24.176523 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:24.178639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:09:24.178639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:09:24.178639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:09:24.178639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:09:24.178639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:09:24.178639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:09:24.178639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:24.178639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:24.178639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 356/710] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:09:24.186007 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030704a5-0af2-4238-9a2b-7ee0373ac8f3 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:24.258192 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864da125-4123-46d5-806c-5b0aafac4e77 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:24.306467 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cde96-f4d5-419c-b29e-ac5a936dce21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:24.307019 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cde96-f4d5-419c-b29e-ac5a936dce21 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:24.307304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cde96-f4d5-419c-b29e-ac5a936dce21 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:24.307583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cde96-f4d5-419c-b29e-ac5a936dce21 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:24.308278 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773cde96-f4d5-419c-b29e-ac5a936dce21 None admin] RBAC: Adding query filter params (name=network) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:24.308585 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773cde96-f4d5-419c-b29e-ac5a936dce21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:24.309952 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773cde96-f4d5-419c-b29e-ac5a936dce21 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:24.323056 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 355/711] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Jul 27 04:09:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:24.328373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cde96-f4d5-419c-b29e-ac5a936dce21 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:24.362481 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:24.385560 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030704a5-0af2-4238-9a2b-7ee0373ac8f3 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:24.442609 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:24.462826 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c148bb2-e343-4440-a801-596b11ef3deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:24.463822 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c148bb2-e343-4440-a801-596b11ef3deb None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:24.464298 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c148bb2-e343-4440-a801-596b11ef3deb None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:24.464762 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c148bb2-e343-4440-a801-596b11ef3deb None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:24.465722 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c148bb2-e343-4440-a801-596b11ef3deb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:24.466288 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c148bb2-e343-4440-a801-596b11ef3deb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:24.467933 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c148bb2-e343-4440-a801-596b11ef3deb None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:24.487655 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 357/712] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:09:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:09:24.495631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c148bb2-e343-4440-a801-596b11ef3deb None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:24.517672 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cde96-f4d5-419c-b29e-ac5a936dce21 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:24.604882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff30e9-49ab-4484-b984-df0a8ce7ce4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:24.605516 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff30e9-49ab-4484-b984-df0a8ce7ce4c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:24.605800 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff30e9-49ab-4484-b984-df0a8ce7ce4c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:24.606101 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff30e9-49ab-4484-b984-df0a8ce7ce4c None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:24.606821 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ff30e9-49ab-4484-b984-df0a8ce7ce4c None admin] RBAC: Adding query filter params (type=network) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:24.607093 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ff30e9-49ab-4484-b984-df0a8ce7ce4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:24.608459 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ff30e9-49ab-4484-b984-df0a8ce7ce4c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:24.619784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c148bb2-e343-4440-a801-596b11ef3deb None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:24.622383 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 356/713] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Jul 27 04:09:24 2022] GET /identity/v3/services?type=network => generated 124 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:24.623913 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9325c4-b508-419b-8227-d97212414d61 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:24.624149 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9325c4-b508-419b-8227-d97212414d61 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:24.624430 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9325c4-b508-419b-8227-d97212414d61 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:24.626658 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 357/714] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:24.647502 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b4c08-c20c-4129-8f36-cba3c3ee7604 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:24.647824 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b4c08-c20c-4129-8f36-cba3c3ee7604 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:24.648347 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b4c08-c20c-4129-8f36-cba3c3ee7604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:24.671387 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72beb044-f0a0-4f9b-aa7e-fe0ac25e8217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:24.672037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72beb044-f0a0-4f9b-aa7e-fe0ac25e8217 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:24.672317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72beb044-f0a0-4f9b-aa7e-fe0ac25e8217 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:24.673371 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72beb044-f0a0-4f9b-aa7e-fe0ac25e8217 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:24.675641 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-b77b4c08-c20c-4129-8f36-cba3c3ee7604 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:24.677121 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b77b4c08-c20c-4129-8f36-cba3c3ee7604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:24.679232 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72beb044-f0a0-4f9b-aa7e-fe0ac25e8217 None admin] RBAC: Adding query filter params (role.id=cf771a8e94ee46149eb081347fea6d67, scope.project.id=e646d320aa5a4f78bfccaf941cf46478, user.id=9dc9ea8f89c14a2a8e3272f750647dfd) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:24.679564 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72beb044-f0a0-4f9b-aa7e-fe0ac25e8217 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:24.681145 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72beb044-f0a0-4f9b-aa7e-fe0ac25e8217 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:24.697941 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 358/715] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:09:24 2022] GET /identity/v3/role_assignments?user.id=9dc9ea8f89c14a2a8e3272f750647dfd&scope.project.id=e646d320aa5a4f78bfccaf941cf46478&role.id=cf771a8e94ee46149eb081347fea6d67 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:24.770901 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 358/716] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:09:24.807274 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72beb044-f0a0-4f9b-aa7e-fe0ac25e8217 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:24.829237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:24.934686 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72beb044-f0a0-4f9b-aa7e-fe0ac25e8217 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:24.974796 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53eaa99-5656-467c-8ac7-b0cfb094705c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:24.975514 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53eaa99-5656-467c-8ac7-b0cfb094705c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:24.975797 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53eaa99-5656-467c-8ac7-b0cfb094705c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:24.976126 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53eaa99-5656-467c-8ac7-b0cfb094705c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:24.988332 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53eaa99-5656-467c-8ac7-b0cfb094705c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:24.989831 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53eaa99-5656-467c-8ac7-b0cfb094705c None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:25.014493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-a53eaa99-5656-467c-8ac7-b0cfb094705c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:25.014493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:25.014493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:25.014493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:25.014493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:25.014493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:25.014493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:25.014493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:25.014493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:25.014493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:25.014493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:25.014493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:25.014493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:25.014493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:25.014493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:25.015447 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:25.015447 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:25.015447 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:25.015447 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:25.015447 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:25.015447 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:25.015447 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:25.015447 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:25.015447 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:25.015447 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:25.015447 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:25.015447 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:25.015447 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:25.015447 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:25.015447 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:25.015447 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:25.015447 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:25.016999 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:25.016999 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:25.016999 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:25.016999 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:25.016999 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 359/717] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:25.024369 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b4c08-c20c-4129-8f36-cba3c3ee7604 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:25.113085 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:25.158165 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b4c08-c20c-4129-8f36-cba3c3ee7604 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:25.194991 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79e40d-bab2-491a-858c-c1bcebad00a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:25.195950 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79e40d-bab2-491a-858c-c1bcebad00a5 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:25.196431 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79e40d-bab2-491a-858c-c1bcebad00a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:25.196934 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79e40d-bab2-491a-858c-c1bcebad00a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:25.197840 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79e40d-bab2-491a-858c-c1bcebad00a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:25.198366 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79e40d-bab2-491a-858c-c1bcebad00a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:25.199923 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79e40d-bab2-491a-858c-c1bcebad00a5 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:25.212464 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 359/718] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:25.217778 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53eaa99-5656-467c-8ac7-b0cfb094705c None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:25.316233 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:25.368603 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53eaa99-5656-467c-8ac7-b0cfb094705c None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:25.415917 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea95602-c9a7-4b50-9383-0593f4158a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:25.416549 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea95602-c9a7-4b50-9383-0593f4158a64 None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:25.416994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea95602-c9a7-4b50-9383-0593f4158a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:25.417280 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea95602-c9a7-4b50-9383-0593f4158a64 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:25.417995 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea95602-c9a7-4b50-9383-0593f4158a64 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:25.419444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea95602-c9a7-4b50-9383-0593f4158a64 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:25.456100 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 360/719] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 27 04:09:25 2022] POST /identity/v3/users => generated 254 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:09:25.976795 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acc838-03f3-4698-ad3f-748908f11cae None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:25.977164 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acc838-03f3-4698-ad3f-748908f11cae None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:25.977366 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acc838-03f3-4698-ad3f-748908f11cae None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:25.979017 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 360/720] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:25.984057 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c10e4a-2ee1-4512-8ef8-f80464d0cba7 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:25.984352 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c10e4a-2ee1-4512-8ef8-f80464d0cba7 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:25.984633 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c10e4a-2ee1-4512-8ef8-f80464d0cba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:26.008553 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-42c10e4a-2ee1-4512-8ef8-f80464d0cba7 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:26.010541 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-42c10e4a-2ee1-4512-8ef8-f80464d0cba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:26.082521 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 361/721] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:09:26.105032 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15acc838-03f3-4698-ad3f-748908f11cae None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:26.208125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31ba1d-a67f-4fe3-85cf-20c68ea69970 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:26.208454 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31ba1d-a67f-4fe3-85cf-20c68ea69970 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:26.208673 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31ba1d-a67f-4fe3-85cf-20c68ea69970 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:26.210159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 362/722] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 04:09:26.215421 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba94da-47c5-4b3f-a952-c6a099810e03 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:26.215984 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba94da-47c5-4b3f-a952-c6a099810e03 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:26.216468 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba94da-47c5-4b3f-a952-c6a099810e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:26.228569 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15acc838-03f3-4698-ad3f-748908f11cae None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:26.244122 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-35ba94da-47c5-4b3f-a952-c6a099810e03 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:26.245361 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-35ba94da-47c5-4b3f-a952-c6a099810e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:26.274352 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:26.322536 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be18f6-fdfa-4e84-ab4f-7d1c640dfac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:26.323174 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be18f6-fdfa-4e84-ab4f-7d1c640dfac3 None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:26.323460 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be18f6-fdfa-4e84-ab4f-7d1c640dfac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:26.323738 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be18f6-fdfa-4e84-ab4f-7d1c640dfac3 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:26.324464 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6be18f6-fdfa-4e84-ab4f-7d1c640dfac3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:26.325845 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6be18f6-fdfa-4e84-ab4f-7d1c640dfac3 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:26.338383 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:26.368327 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 361/723] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Jul 27 04:09:26 2022] POST /identity/v3/services => generated 251 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:09:26.370603 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 363/724] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 04:09:26.393604 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be18f6-fdfa-4e84-ab4f-7d1c640dfac3 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:26.555493 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be18f6-fdfa-4e84-ab4f-7d1c640dfac3 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:26.598149 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a436d3c-3d24-42e0-8d69-b95b65d1704a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:26.598808 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a436d3c-3d24-42e0-8d69-b95b65d1704a None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:26.599145 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a436d3c-3d24-42e0-8d69-b95b65d1704a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:26.599426 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a436d3c-3d24-42e0-8d69-b95b65d1704a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:26.611650 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a436d3c-3d24-42e0-8d69-b95b65d1704a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:09:26.617440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-8a436d3c-3d24-42e0-8d69-b95b65d1704a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:09:26.617440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:26.617440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:26.617440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:26.617440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:26.617440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:26.617440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:26.617440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:26.617440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:26.617440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:26.617440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:26.617440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:26.617440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:09:26.617440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:09:26.617440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:09:26.617440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:09:26.618407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:09:26.618407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:09:26.618407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:09:26.618407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:09:26.618407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:26.618407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:26.618407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:26.618407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:26.618407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:26.618407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:26.618407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:26.618407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:26.618407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:26.618407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:26.618407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:26.618407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:26.619229 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:26.619229 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:26.619229 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:26.619229 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:26.619229 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:09:26.619229 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:09:26.619229 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:09:26.619229 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:09:26.619229 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:09:26.619229 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:09:26.619229 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:09:26.619229 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:26.621877 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 362/725] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:09:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:26.628085 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba94da-47c5-4b3f-a952-c6a099810e03 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:26.758355 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba94da-47c5-4b3f-a952-c6a099810e03 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:26.809775 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867c250-0d40-461f-bb42-3924f5f58d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:26.810784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867c250-0d40-461f-bb42-3924f5f58d4f None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:26.811377 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867c250-0d40-461f-bb42-3924f5f58d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:26.811877 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867c250-0d40-461f-bb42-3924f5f58d4f None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:26.812809 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6867c250-0d40-461f-bb42-3924f5f58d4f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:26.813720 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6867c250-0d40-461f-bb42-3924f5f58d4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:26.815337 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6867c250-0d40-461f-bb42-3924f5f58d4f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:26.830033 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 364/726] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:09:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:26.835649 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a436d3c-3d24-42e0-8d69-b95b65d1704a None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:26.869046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:26.941819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a436d3c-3d24-42e0-8d69-b95b65d1704a None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:26.977389 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c643a9-9b38-421b-8bbc-c027e200e550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:26.978470 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c643a9-9b38-421b-8bbc-c027e200e550 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:26.979076 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c643a9-9b38-421b-8bbc-c027e200e550 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:26.979566 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c643a9-9b38-421b-8bbc-c027e200e550 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:26.990833 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c643a9-9b38-421b-8bbc-c027e200e550 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:26.992556 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c643a9-9b38-421b-8bbc-c027e200e550 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:27.007950 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-09c643a9-9b38-421b-8bbc-c027e200e550 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:27.007950 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:27.007950 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:27.007950 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:27.007950 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:27.007950 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:27.007950 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:27.007950 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:27.007950 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:27.007950 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:27.007950 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:27.007950 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:27.007950 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:27.007950 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:27.007950 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:27.008875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:27.008875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:27.008875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:27.008875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:27.008875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:27.008875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:27.008875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:27.008875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:27.008875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:27.008875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:27.008875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:27.008875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:27.008875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:27.008875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:27.008875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:27.008875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:27.008875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:27.009754 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:27.009754 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:27.009754 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:27.009754 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:27.011489 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 363/727] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:27.017436 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867c250-0d40-461f-bb42-3924f5f58d4f None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:27.083678 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:27.120581 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531b6ae-7773-44c9-a9d0-8cd3fcc455c4 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:27.121127 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531b6ae-7773-44c9-a9d0-8cd3fcc455c4 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:27.121127 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531b6ae-7773-44c9-a9d0-8cd3fcc455c4 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:27.123013 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 364/728] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:27.128118 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100fd2c-cfbe-49fc-8746-e54e11688bde None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:27.128406 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100fd2c-cfbe-49fc-8746-e54e11688bde None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:27.128689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100fd2c-cfbe-49fc-8746-e54e11688bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:27.167821 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867c250-0d40-461f-bb42-3924f5f58d4f None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:27.175070 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-d100fd2c-cfbe-49fc-8746-e54e11688bde None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:27.176304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d100fd2c-cfbe-49fc-8746-e54e11688bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:27.207025 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e26e92-7631-465a-8a5a-89c26f240971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:27.208172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e26e92-7631-465a-8a5a-89c26f240971 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:27.208663 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e26e92-7631-465a-8a5a-89c26f240971 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:27.209123 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e26e92-7631-465a-8a5a-89c26f240971 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:27.210053 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e26e92-7631-465a-8a5a-89c26f240971 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:27.210543 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e26e92-7631-465a-8a5a-89c26f240971 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:27.212733 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e26e92-7631-465a-8a5a-89c26f240971 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:27.226991 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 365/729] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:27.232767 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e26e92-7631-465a-8a5a-89c26f240971 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:27.346634 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 365/730] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:27 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 04:09:27.347901 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac9ae8-1105-486a-ba37-4c83e50f6f95 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:27.348191 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac9ae8-1105-486a-ba37-4c83e50f6f95 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:27.348468 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac9ae8-1105-486a-ba37-4c83e50f6f95 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:27.349909 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 366/731] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:27.357458 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e26e92-7631-465a-8a5a-89c26f240971 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:27.367783 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167a7ea-d079-409e-9d2f-071585d0608f None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:27.368052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167a7ea-d079-409e-9d2f-071585d0608f None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:27.368378 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167a7ea-d079-409e-9d2f-071585d0608f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:27.373684 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:27.388345 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-d167a7ea-d079-409e-9d2f-071585d0608f None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:27.389651 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d167a7ea-d079-409e-9d2f-071585d0608f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:27.400372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:27.407795 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48fbd3-08ea-4c5d-8127-59bfa02c9a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:27.408518 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48fbd3-08ea-4c5d-8127-59bfa02c9a2b None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:27.408812 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48fbd3-08ea-4c5d-8127-59bfa02c9a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:27.409082 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48fbd3-08ea-4c5d-8127-59bfa02c9a2b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:27.416876 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d48fbd3-08ea-4c5d-8127-59bfa02c9a2b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:27.418410 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d48fbd3-08ea-4c5d-8127-59bfa02c9a2b None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:27.431527 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-1d48fbd3-08ea-4c5d-8127-59bfa02c9a2b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 04:09:27.431527 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:27.431527 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:27.431527 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:27.431527 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:27.431527 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:27.431527 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:27.431527 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:27.431527 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:27.431527 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:27.431527 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:27.431527 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:27.431527 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:09:27.431527 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:09:27.431527 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:09:27.431527 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:09:27.432610 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:27.432610 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:27.432610 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:09:27.432610 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:27.432610 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:09:27.432610 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:27.432610 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:27.432610 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:27.432610 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:27.432610 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:27.432610 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:27.432610 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:27.432610 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:27.432610 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:27.432610 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:27.432610 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:27.432610 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:27.432610 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:27.433591 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:27.433591 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:27.433591 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:09:27.433591 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:09:27.433591 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:09:27.433591 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:09:27.433591 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:09:27.433591 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:09:27.433591 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 04:09:27.433591 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:27.433591 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 366/732] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:09:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:27.434245 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48fbd3-08ea-4c5d-8127-59bfa02c9a2b None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:27.486820 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 367/733] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:27 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 04:09:27.488013 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167a7ea-d079-409e-9d2f-071585d0608f None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:27.591009 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48fbd3-08ea-4c5d-8127-59bfa02c9a2b None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:27.622585 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167a7ea-d079-409e-9d2f-071585d0608f None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:27.627073 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2282b-41b3-4fa1-a507-9d28f4e4f7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:27.627968 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2282b-41b3-4fa1-a507-9d28f4e4f7ec None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:27.628444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2282b-41b3-4fa1-a507-9d28f4e4f7ec None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:27.628910 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2282b-41b3-4fa1-a507-9d28f4e4f7ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:27.639830 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab2282b-41b3-4fa1-a507-9d28f4e4f7ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:09:27.645598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-5ab2282b-41b3-4fa1-a507-9d28f4e4f7ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:09:27.645598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:27.645598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:27.645598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:27.645598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:27.645598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:27.645598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:27.645598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:27.645598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:27.645598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:27.645598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:27.645598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:27.645598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:09:27.645598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:09:27.645598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:09:27.645598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:09:27.646724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:09:27.646724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:09:27.646724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:09:27.646724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:09:27.646724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:27.646724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:27.646724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:27.646724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:27.646724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:27.646724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:27.646724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:27.646724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:27.646724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:27.646724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:27.646724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:27.646724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:27.647596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:27.647596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:27.647596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:27.647596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:27.647596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:09:27.647596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:09:27.647596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:09:27.647596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:09:27.647596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:09:27.647596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:09:27.647596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:09:27.647596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:27.654651 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 367/734] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:09:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:27.655651 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2282b-41b3-4fa1-a507-9d28f4e4f7ec None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:27.661560 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff1ecad-dc0f-41fb-b80a-adb87e2ee2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:27.662260 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1ecad-dc0f-41fb-b80a-adb87e2ee2c3 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:27.662546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1ecad-dc0f-41fb-b80a-adb87e2ee2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:27.662822 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1ecad-dc0f-41fb-b80a-adb87e2ee2c3 None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:27.663633 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff1ecad-dc0f-41fb-b80a-adb87e2ee2c3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:27.663939 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff1ecad-dc0f-41fb-b80a-adb87e2ee2c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:27.665434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff1ecad-dc0f-41fb-b80a-adb87e2ee2c3 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:27.686578 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 368/735] 10.222.0.46 () {56 vars in 1135 bytes} [Wed Jul 27 04:09:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:27.687722 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff1ecad-dc0f-41fb-b80a-adb87e2ee2c3 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:27.691934 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:27.695141 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:27.788745 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2282b-41b3-4fa1-a507-9d28f4e4f7ec None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:27.814148 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff1ecad-dc0f-41fb-b80a-adb87e2ee2c3 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:27.825359 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c2b02b-5b20-4cd8-bcd2-8bb8e867cf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:27.826318 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2b02b-5b20-4cd8-bcd2-8bb8e867cf3c None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:27.826798 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2b02b-5b20-4cd8-bcd2-8bb8e867cf3c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:27.827265 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2b02b-5b20-4cd8-bcd2-8bb8e867cf3c None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:27.828169 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c2b02b-5b20-4cd8-bcd2-8bb8e867cf3c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:27.829913 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c2b02b-5b20-4cd8-bcd2-8bb8e867cf3c None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:27.851008 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 368/736] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Jul 27 04:09:27 2022] POST /identity/v3/services => generated 240 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:09:27.852265 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c2b02b-5b20-4cd8-bcd2-8bb8e867cf3c None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:27.882876 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01905b61-223c-4c6f-9bb1-1d26149b15ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:27.883459 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01905b61-223c-4c6f-9bb1-1d26149b15ce None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:27.883725 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01905b61-223c-4c6f-9bb1-1d26149b15ce None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:27.884005 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01905b61-223c-4c6f-9bb1-1d26149b15ce None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:27.884747 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01905b61-223c-4c6f-9bb1-1d26149b15ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:27.885074 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01905b61-223c-4c6f-9bb1-1d26149b15ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:27.886462 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01905b61-223c-4c6f-9bb1-1d26149b15ce None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:27.909273 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 369/737] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:09:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:27.925889 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01905b61-223c-4c6f-9bb1-1d26149b15ce None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:27.970372 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:27.996938 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c2b02b-5b20-4cd8-bcd2-8bb8e867cf3c None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:28.041150 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca789c-ab21-424d-ba72-c193fe6394e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:28.042710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca789c-ab21-424d-ba72-c193fe6394e2 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:28.043715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca789c-ab21-424d-ba72-c193fe6394e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:28.044230 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca789c-ab21-424d-ba72-c193fe6394e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:28.087502 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:28.132159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca789c-ab21-424d-ba72-c193fe6394e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:28.134952 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca789c-ab21-424d-ba72-c193fe6394e2 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:28.139392 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01905b61-223c-4c6f-9bb1-1d26149b15ce None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:28.177361 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce427c5-2770-47f7-8b2d-9dedc8bbba59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:28.178150 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce427c5-2770-47f7-8b2d-9dedc8bbba59 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:28.178431 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce427c5-2770-47f7-8b2d-9dedc8bbba59 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:28.178713 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce427c5-2770-47f7-8b2d-9dedc8bbba59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:28.191276 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce427c5-2770-47f7-8b2d-9dedc8bbba59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:28.192675 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce427c5-2770-47f7-8b2d-9dedc8bbba59 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:28.209144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-cce427c5-2770-47f7-8b2d-9dedc8bbba59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:28.209144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:28.209144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:28.209144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:28.209144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:28.209144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:28.209144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:28.209144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:28.209144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:28.209144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:28.209144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:28.209144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:28.209144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:28.209144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:28.209144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:28.210095 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:28.210095 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:28.210095 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:28.210095 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:28.210095 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:28.210095 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:28.210095 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:28.210095 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:28.210095 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:28.210095 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:28.210095 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:28.210095 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:28.210095 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:28.210095 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:28.210095 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:28.210095 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:28.210095 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:28.211001 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:28.211001 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:28.211001 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:28.211001 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:28.211001 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 370/738] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:09:28.216124 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce427c5-2770-47f7-8b2d-9dedc8bbba59 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:28.242010 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:28.268953 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 369/739] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:09:27 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 04:09:28.275192 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca789c-ab21-424d-ba72-c193fe6394e2 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:28.345905 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce427c5-2770-47f7-8b2d-9dedc8bbba59 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:28.361067 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:28.394674 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4245ed2-6b81-42f2-b996-e3f97113e19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:28.395341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4245ed2-6b81-42f2-b996-e3f97113e19e None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:28.395685 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4245ed2-6b81-42f2-b996-e3f97113e19e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:28.396033 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4245ed2-6b81-42f2-b996-e3f97113e19e None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:28.396896 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4245ed2-6b81-42f2-b996-e3f97113e19e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:28.397272 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4245ed2-6b81-42f2-b996-e3f97113e19e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:28.398928 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4245ed2-6b81-42f2-b996-e3f97113e19e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:28.412571 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 371/740] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:09:28.414058 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3554e-4740-4123-8a4f-782b6fca63d5 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:28.417834 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3554e-4740-4123-8a4f-782b6fca63d5 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:28.417834 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3554e-4740-4123-8a4f-782b6fca63d5 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:28.418049 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca789c-ab21-424d-ba72-c193fe6394e2 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:28.418129 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 372/741] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:28.418844 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3554e-4740-4123-8a4f-782b6fca63d5 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:28.460174 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e0964-1f43-4e4f-9485-9cbc87fa4794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:28.461054 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e0964-1f43-4e4f-9485-9cbc87fa4794 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:28.461530 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e0964-1f43-4e4f-9485-9cbc87fa4794 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:28.467398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e0964-1f43-4e4f-9485-9cbc87fa4794 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:28.482875 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9e0964-1f43-4e4f-9485-9cbc87fa4794 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:28.484736 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9e0964-1f43-4e4f-9485-9cbc87fa4794 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:28.502299 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-0c9e0964-1f43-4e4f-9485-9cbc87fa4794 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:28.502299 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:28.502299 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:28.502299 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:28.502299 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:28.502299 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:28.502299 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:28.502299 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:28.502299 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:28.502299 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:28.502299 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:28.502299 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:28.502299 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:28.502299 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:28.502299 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:28.503253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:28.503253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:28.503253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:28.503253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:28.503253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:28.503253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:28.503253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:28.503253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:28.503253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:28.503253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:28.503253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:28.503253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:28.503253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:28.503253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:28.503253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:28.503253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:28.503253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:28.504134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:28.504134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:28.504134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:28.504134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:28.511160 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 370/742] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:28.512724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae4d57-8f97-4b50-b0b7-5165cbe00774 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:28.513222 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae4d57-8f97-4b50-b0b7-5165cbe00774 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:28.513704 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae4d57-8f97-4b50-b0b7-5165cbe00774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:28.531607 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3554e-4740-4123-8a4f-782b6fca63d5 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:28.533213 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-02ae4d57-8f97-4b50-b0b7-5165cbe00774 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:28.534663 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-02ae4d57-8f97-4b50-b0b7-5165cbe00774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:28.562398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:28.572321 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0311bce6-5d03-48d5-a026-24807002e0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:28.573251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311bce6-5d03-48d5-a026-24807002e0b1 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:28.573675 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311bce6-5d03-48d5-a026-24807002e0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:28.574142 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311bce6-5d03-48d5-a026-24807002e0b1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:28.582501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0311bce6-5d03-48d5-a026-24807002e0b1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:28.584391 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0311bce6-5d03-48d5-a026-24807002e0b1 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:28.598256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-0311bce6-5d03-48d5-a026-24807002e0b1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 04:09:28.598256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:28.598256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:28.598256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:28.598256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:28.598256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:28.598256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:28.598256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:28.598256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:28.598256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:28.598256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:28.598256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:28.598256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:09:28.598256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:09:28.598256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:09:28.598256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:09:28.599373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:28.599373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:28.599373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:09:28.599373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:28.599373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:09:28.599373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:28.599373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:28.599373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:28.599373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:28.599373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:28.599373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:28.599373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:28.599373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:28.599373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:28.599373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:28.599373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:28.599373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:28.599373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:28.600472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:28.600472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:28.600472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:09:28.600472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:09:28.600472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:09:28.600472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:09:28.600472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:09:28.600472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:09:28.600472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 04:09:28.600472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:28.602679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 373/743] 10.222.0.46 () {56 vars in 1093 bytes} [Wed Jul 27 04:09:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:28.603934 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0311bce6-5d03-48d5-a026-24807002e0b1 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:28.614295 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 371/744] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:09:28.615258 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae4d57-8f97-4b50-b0b7-5165cbe00774 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:28.639027 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:28.714221 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0311bce6-5d03-48d5-a026-24807002e0b1 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:28.732009 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae4d57-8f97-4b50-b0b7-5165cbe00774 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:28.751120 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd1dd6-cb0e-4773-8334-e4d136022967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:28.752240 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd1dd6-cb0e-4773-8334-e4d136022967 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:28.752748 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd1dd6-cb0e-4773-8334-e4d136022967 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:28.753244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd1dd6-cb0e-4773-8334-e4d136022967 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:28.754174 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd1dd6-cb0e-4773-8334-e4d136022967 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:28.754734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd1dd6-cb0e-4773-8334-e4d136022967 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:28.756478 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd1dd6-cb0e-4773-8334-e4d136022967 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:28.767483 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399c1e0-2d61-4ded-a2a8-1e472506ca81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:28.767828 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399c1e0-2d61-4ded-a2a8-1e472506ca81 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:28.768100 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399c1e0-2d61-4ded-a2a8-1e472506ca81 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:28.768366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399c1e0-2d61-4ded-a2a8-1e472506ca81 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:28.769256 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d399c1e0-2d61-4ded-a2a8-1e472506ca81 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:28.769465 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d399c1e0-2d61-4ded-a2a8-1e472506ca81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:28.770917 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d399c1e0-2d61-4ded-a2a8-1e472506ca81 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:28.772280 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 374/745] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:28.773511 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd1dd6-cb0e-4773-8334-e4d136022967 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:28.789456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 372/746] 10.222.0.46 () {56 vars in 1133 bytes} [Wed Jul 27 04:09:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:28.790187 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399c1e0-2d61-4ded-a2a8-1e472506ca81 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:28.815098 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:28.874468 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:28.886945 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd1dd6-cb0e-4773-8334-e4d136022967 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:28.894174 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399c1e0-2d61-4ded-a2a8-1e472506ca81 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:28.930104 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189dd72f-a005-42a6-ba0e-915bd6802d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:28.930794 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189dd72f-a005-42a6-ba0e-915bd6802d2d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:28.931032 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189dd72f-a005-42a6-ba0e-915bd6802d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:28.931308 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189dd72f-a005-42a6-ba0e-915bd6802d2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:28.935395 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b538ab-c1fc-44e6-8cb4-0b6f7b9f6f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:28.936038 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b538ab-c1fc-44e6-8cb4-0b6f7b9f6f22 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:28.936313 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b538ab-c1fc-44e6-8cb4-0b6f7b9f6f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:28.936582 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b538ab-c1fc-44e6-8cb4-0b6f7b9f6f22 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:28.939162 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189dd72f-a005-42a6-ba0e-915bd6802d2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:28.940615 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189dd72f-a005-42a6-ba0e-915bd6802d2d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:28.947948 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b538ab-c1fc-44e6-8cb4-0b6f7b9f6f22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:09:28.951999 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-189dd72f-a005-42a6-ba0e-915bd6802d2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:28.951999 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:28.951999 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:28.951999 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:28.951999 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:28.951999 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:28.951999 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:28.951999 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:28.951999 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:28.951999 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:28.951999 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:28.951999 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:28.951999 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:09:28.951999 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:09:28.951999 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:09:28.952878 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:09:28.952878 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:28.952878 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:28.952878 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:28.952878 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:28.952878 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:28.952878 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:28.952878 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:28.952878 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:28.952878 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:28.952878 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:28.952878 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:28.952878 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:28.952878 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:28.952878 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:28.952878 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:28.952878 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:28.953883 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:09:28.953883 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:09:28.953883 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:09:28.953883 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:09:28.953883 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:09:28.953883 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:09:28.953883 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:28.953883 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:28.954346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-f2b538ab-c1fc-44e6-8cb4-0b6f7b9f6f22 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:28.954346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:28.954346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:28.954346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:28.954346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:28.954346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:28.954346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:28.954346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:28.954346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:28.954346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:28.954346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:28.954346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:28.954346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:09:28.954346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:09:28.954346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:09:28.954346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:09:28.955163 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:09:28.955163 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:09:28.955163 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:09:28.955163 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:09:28.955163 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:28.955163 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:28.955163 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:28.955163 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:28.955163 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:28.955163 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:28.955163 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:28.955163 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:28.955163 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:28.955163 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:28.955163 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:28.955163 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:28.955994 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:28.955994 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:28.955994 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:28.955994 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:28.955994 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:09:28.955994 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:09:28.955994 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:09:28.955994 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:09:28.955994 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:09:28.955994 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:09:28.955994 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:28.955994 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:28.957853 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 373/747] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:28.958327 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 375/748] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 27 04:09:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:28.958737 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189dd72f-a005-42a6-ba0e-915bd6802d2d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:28.960032 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a3b7d-6166-4db9-b9f0-0f066a1e3053 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:28.960321 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a3b7d-6166-4db9-b9f0-0f066a1e3053 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:28.960601 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a3b7d-6166-4db9-b9f0-0f066a1e3053 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:28.963055 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 376/749] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:28.963774 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a3b7d-6166-4db9-b9f0-0f066a1e3053 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:29.042623 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:29.064440 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189dd72f-a005-42a6-ba0e-915bd6802d2d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:29.076882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a3b7d-6166-4db9-b9f0-0f066a1e3053 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:29.099811 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b920489-372b-4ca3-a0b2-bb02dae4c98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:29.100745 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b920489-372b-4ca3-a0b2-bb02dae4c98f None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:29.101256 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b920489-372b-4ca3-a0b2-bb02dae4c98f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:29.101713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b920489-372b-4ca3-a0b2-bb02dae4c98f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:29.114211 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b920489-372b-4ca3-a0b2-bb02dae4c98f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:29.115025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf01937-93c8-4fc2-ab5c-0ee810c02ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:29.115644 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf01937-93c8-4fc2-ab5c-0ee810c02ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:29.115924 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf01937-93c8-4fc2-ab5c-0ee810c02ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:29.116198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf01937-93c8-4fc2-ab5c-0ee810c02ed0 None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:29.116560 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b920489-372b-4ca3-a0b2-bb02dae4c98f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:29.116949 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf01937-93c8-4fc2-ab5c-0ee810c02ed0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:29.117253 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf01937-93c8-4fc2-ab5c-0ee810c02ed0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:29.118680 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf01937-93c8-4fc2-ab5c-0ee810c02ed0 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:29.131056 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 377/750] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:09:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:29.131740 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf01937-93c8-4fc2-ab5c-0ee810c02ed0 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:29.134115 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-7b920489-372b-4ca3-a0b2-bb02dae4c98f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:29.134115 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:29.134115 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:29.134115 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:29.134115 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:29.134115 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:29.134115 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:29.134115 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:29.134115 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:29.134115 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:29.134115 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:29.134115 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:29.134115 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:29.134115 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:29.134115 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:29.135002 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:29.135002 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:29.135002 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:29.135002 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:29.135002 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:29.135002 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:29.135002 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:29.135002 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:29.135002 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:29.135002 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:29.135002 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:29.135002 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:29.135002 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:29.135002 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:29.135002 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:29.135002 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:29.135002 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:29.135944 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:29.135944 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:29.135944 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:29.135944 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:29.135944 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 374/751] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:29.136800 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a72b4-5d3f-467e-8cbc-96a3430474ad None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:29.137141 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a72b4-5d3f-467e-8cbc-96a3430474ad None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:29.137418 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a72b4-5d3f-467e-8cbc-96a3430474ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:29.153841 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:29.159048 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-724a72b4-5d3f-467e-8cbc-96a3430474ad None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:29.160239 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-724a72b4-5d3f-467e-8cbc-96a3430474ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:29.246154 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 375/752] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:09:29.246154 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a72b4-5d3f-467e-8cbc-96a3430474ad None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:29.249819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf01937-93c8-4fc2-ab5c-0ee810c02ed0 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:29.289259 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ebbf1-6187-494c-b86d-fb49f63de06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:29.289888 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ebbf1-6187-494c-b86d-fb49f63de06e None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:29.290205 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ebbf1-6187-494c-b86d-fb49f63de06e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:29.290479 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ebbf1-6187-494c-b86d-fb49f63de06e None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:29.291198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071ebbf1-6187-494c-b86d-fb49f63de06e None admin] RBAC: Adding query filter params (name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:29.291504 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071ebbf1-6187-494c-b86d-fb49f63de06e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:29.292919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071ebbf1-6187-494c-b86d-fb49f63de06e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:29.301707 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 378/753] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 27 04:09:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:29.307069 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ebbf1-6187-494c-b86d-fb49f63de06e None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:29.321954 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:29.367075 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a72b4-5d3f-467e-8cbc-96a3430474ad None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:29.387406 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:29.405346 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f4d45-e913-4aab-9dd6-7692f32ada00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:29.406002 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f4d45-e913-4aab-9dd6-7692f32ada00 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:29.406274 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f4d45-e913-4aab-9dd6-7692f32ada00 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:29.406543 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f4d45-e913-4aab-9dd6-7692f32ada00 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:29.407790 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772f4d45-e913-4aab-9dd6-7692f32ada00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:29.408092 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772f4d45-e913-4aab-9dd6-7692f32ada00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:29.409766 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772f4d45-e913-4aab-9dd6-7692f32ada00 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:29.418436 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ebbf1-6187-494c-b86d-fb49f63de06e None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:29.422500 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 376/754] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:29.424041 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f4d45-e913-4aab-9dd6-7692f32ada00 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:29.455034 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f4374-c9ee-43ea-940b-9f9139e72066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:29.455655 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f4374-c9ee-43ea-940b-9f9139e72066 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:29.455937 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f4374-c9ee-43ea-940b-9f9139e72066 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:29.456213 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f4374-c9ee-43ea-940b-9f9139e72066 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:29.457003 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2f4374-c9ee-43ea-940b-9f9139e72066 None admin] RBAC: Adding query filter params (role.id=cf771a8e94ee46149eb081347fea6d67, scope.project.id=e646d320aa5a4f78bfccaf941cf46478, user.id=57b5fa7517034a39930ce7c9d12e1875) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:29.457313 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2f4374-c9ee-43ea-940b-9f9139e72066 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:29.458757 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2f4374-c9ee-43ea-940b-9f9139e72066 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:29.467609 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 379/755] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:09:29 2022] GET /identity/v3/role_assignments?user.id=57b5fa7517034a39930ce7c9d12e1875&scope.project.id=e646d320aa5a4f78bfccaf941cf46478&role.id=cf771a8e94ee46149eb081347fea6d67 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:29.468424 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f4374-c9ee-43ea-940b-9f9139e72066 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:29.544961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f4d45-e913-4aab-9dd6-7692f32ada00 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:29.578761 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:29.588637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e929a-9e18-4c93-9bf7-cfe4b857619d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:29.589310 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e929a-9e18-4c93-9bf7-cfe4b857619d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:29.589593 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e929a-9e18-4c93-9bf7-cfe4b857619d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:29.589869 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e929a-9e18-4c93-9bf7-cfe4b857619d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:29.590547 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2e929a-9e18-4c93-9bf7-cfe4b857619d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:29.593501 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2e929a-9e18-4c93-9bf7-cfe4b857619d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:29.607111 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-4c2e929a-9e18-4c93-9bf7-cfe4b857619d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 04:09:29.607111 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:29.607111 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 04:09:29.607111 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 04:09:29.607111 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 04:09:29.607111 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 04:09:29.607111 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 04:09:29.607111 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:29.607111 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 04:09:29.607111 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:29.607111 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 04:09:29.607111 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:29.607111 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:29.607111 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:29.607111 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:29.607111 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:29.608159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:29.608159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:29.608159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:29.608159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:29.608159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:29.608159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:29.608159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:29.608159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 04:09:29.608159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 04:09:29.608159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 04:09:29.608159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 04:09:29.608159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:29.608159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:29.608159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:29.608159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:29.608159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:29.608995 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:29.608995 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:29.608995 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:29.608995 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:29.608995 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:29.608995 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:29.608995 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:29.608995 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:29.608995 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:29.608995 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:29.608995 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:29.608995 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 04:09:29.608995 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:29.608995 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 04:09:29.608995 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:29.608995 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 377/756] 10.222.0.46 () {56 vars in 1129 bytes} [Wed Jul 27 04:09:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:29.610411 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e929a-9e18-4c93-9bf7-cfe4b857619d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:29.622083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f4374-c9ee-43ea-940b-9f9139e72066 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:29.665310 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee0ab8c-fb6a-4f54-9bb1-8175c7ddc562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:29.666500 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0ab8c-fb6a-4f54-9bb1-8175c7ddc562 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:29.666930 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0ab8c-fb6a-4f54-9bb1-8175c7ddc562 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:29.667370 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0ab8c-fb6a-4f54-9bb1-8175c7ddc562 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:29.677308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:29.682912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee0ab8c-fb6a-4f54-9bb1-8175c7ddc562 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:29.685173 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee0ab8c-fb6a-4f54-9bb1-8175c7ddc562 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:29.704712 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-5ee0ab8c-fb6a-4f54-9bb1-8175c7ddc562 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:29.704712 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:29.704712 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:29.704712 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:29.704712 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:29.704712 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:29.704712 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:29.704712 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:29.704712 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:29.704712 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:29.704712 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:29.704712 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:29.704712 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:29.704712 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:29.704712 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:29.705907 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:29.705907 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:29.705907 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:29.705907 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:29.705907 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:29.705907 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:29.705907 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:29.705907 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:29.705907 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:29.705907 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:29.705907 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:29.705907 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:29.705907 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:29.705907 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:29.705907 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:29.705907 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:29.705907 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:29.707025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:29.707025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:29.707025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:29.707025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:29.709206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 380/757] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:29.710192 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee0ab8c-fb6a-4f54-9bb1-8175c7ddc562 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:29.730701 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e929a-9e18-4c93-9bf7-cfe4b857619d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:29.775269 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4104301-dc51-4f32-8527-0518a611f902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:29.776065 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4104301-dc51-4f32-8527-0518a611f902 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:29.776444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4104301-dc51-4f32-8527-0518a611f902 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:29.776816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4104301-dc51-4f32-8527-0518a611f902 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:29.784299 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4104301-dc51-4f32-8527-0518a611f902 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:29.785869 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4104301-dc51-4f32-8527-0518a611f902 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:29.798881 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-a4104301-dc51-4f32-8527-0518a611f902 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:29.798881 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:29.798881 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:29.798881 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:29.798881 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:29.798881 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:29.798881 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:29.798881 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:29.798881 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:29.798881 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:29.798881 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:29.798881 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:29.798881 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:09:29.798881 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:09:29.798881 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:09:29.799969 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:09:29.799969 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:29.799969 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:29.799969 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:29.799969 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:29.799969 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:29.799969 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:29.799969 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:29.799969 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:29.799969 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:29.799969 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:29.799969 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:29.799969 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:29.799969 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:29.799969 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:29.799969 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:29.799969 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:29.801080 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:09:29.801080 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:09:29.801080 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:09:29.801080 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:09:29.801080 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:09:29.801080 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:09:29.801080 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:29.801080 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:29.807232 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 378/758] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:29.808244 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4104301-dc51-4f32-8527-0518a611f902 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:29.842425 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee0ab8c-fb6a-4f54-9bb1-8175c7ddc562 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:29.864385 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:29.894772 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db17373-babe-411f-8b69-366c58aab59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:29.896155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db17373-babe-411f-8b69-366c58aab59b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:29.896620 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db17373-babe-411f-8b69-366c58aab59b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:29.897047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db17373-babe-411f-8b69-366c58aab59b None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:29.898663 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db17373-babe-411f-8b69-366c58aab59b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:29.899125 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db17373-babe-411f-8b69-366c58aab59b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:29.900947 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db17373-babe-411f-8b69-366c58aab59b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:29.909516 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 381/759] 10.222.0.46 () {56 vars in 1124 bytes} [Wed Jul 27 04:09:29 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:29.910875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db17373-babe-411f-8b69-366c58aab59b None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:29.923265 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4104301-dc51-4f32-8527-0518a611f902 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:29.960250 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38cb73-aca9-4741-93ea-cd6abcb58d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:29.961011 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38cb73-aca9-4741-93ea-cd6abcb58d4e None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:29.961404 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38cb73-aca9-4741-93ea-cd6abcb58d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:29.961753 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38cb73-aca9-4741-93ea-cd6abcb58d4e None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:29.963178 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e38cb73-aca9-4741-93ea-cd6abcb58d4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:29.963178 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e38cb73-aca9-4741-93ea-cd6abcb58d4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:29.964464 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e38cb73-aca9-4741-93ea-cd6abcb58d4e None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:29.983606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 379/760] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:29.984654 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38cb73-aca9-4741-93ea-cd6abcb58d4e None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:29.989087 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:30.041746 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db17373-babe-411f-8b69-366c58aab59b None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:30.078742 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124876c-086e-40ce-b145-c92358457697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:30.079508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124876c-086e-40ce-b145-c92358457697 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:30.079858 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124876c-086e-40ce-b145-c92358457697 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:30.080250 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124876c-086e-40ce-b145-c92358457697 None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:30.081114 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4124876c-086e-40ce-b145-c92358457697 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:30.081521 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4124876c-086e-40ce-b145-c92358457697 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:30.083220 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4124876c-086e-40ce-b145-c92358457697 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:30.100019 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38cb73-aca9-4741-93ea-cd6abcb58d4e None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:30.102283 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 382/761] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:09:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:30.103073 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124876c-086e-40ce-b145-c92358457697 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:30.112950 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:30.140348 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f46dfe-d008-4d00-8390-2c8bc36abbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:30.141514 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f46dfe-d008-4d00-8390-2c8bc36abbb2 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:30.142563 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f46dfe-d008-4d00-8390-2c8bc36abbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:30.143047 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f46dfe-d008-4d00-8390-2c8bc36abbb2 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:30.143956 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f46dfe-d008-4d00-8390-2c8bc36abbb2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:30.144464 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f46dfe-d008-4d00-8390-2c8bc36abbb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:30.146097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f46dfe-d008-4d00-8390-2c8bc36abbb2 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:30.153016 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 380/762] 10.222.0.46 () {56 vars in 1124 bytes} [Wed Jul 27 04:09:29 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:30.154169 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f46dfe-d008-4d00-8390-2c8bc36abbb2 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:30.216167 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124876c-086e-40ce-b145-c92358457697 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:30.232223 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:30.256721 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b902c-8222-425a-8ad8-798e6c217cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:30.257623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b902c-8222-425a-8ad8-798e6c217cfc None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:30.258079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b902c-8222-425a-8ad8-798e6c217cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:30.258499 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b902c-8222-425a-8ad8-798e6c217cfc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:30.264359 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f46dfe-d008-4d00-8390-2c8bc36abbb2 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:30.267476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b902c-8222-425a-8ad8-798e6c217cfc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:30.268965 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b902c-8222-425a-8ad8-798e6c217cfc None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:30.283260 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-094b902c-8222-425a-8ad8-798e6c217cfc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 04:09:30.283260 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:30.283260 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:30.283260 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:30.283260 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:30.283260 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:30.283260 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:30.283260 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:30.283260 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:30.283260 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:30.283260 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:30.283260 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:30.283260 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:09:30.283260 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:09:30.283260 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:09:30.283260 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:09:30.284219 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:30.284219 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:30.284219 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:09:30.284219 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:30.284219 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:09:30.284219 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:30.284219 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:30.284219 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:30.284219 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:30.284219 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:30.284219 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:30.284219 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:30.284219 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:30.284219 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:30.284219 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:30.284219 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:30.284219 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:30.284219 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:30.285143 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:30.285143 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:30.285143 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:09:30.285143 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:09:30.285143 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:09:30.285143 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:09:30.285143 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:09:30.285143 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:09:30.285143 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 04:09:30.285143 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:30.285143 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 383/763] 10.222.0.46 () {56 vars in 1117 bytes} [Wed Jul 27 04:09:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:30.285722 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b902c-8222-425a-8ad8-798e6c217cfc None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:30.307904 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3719979-ab92-4946-867c-c586a033cfd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:30.308577 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3719979-ab92-4946-867c-c586a033cfd0 None admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:30.308863 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3719979-ab92-4946-867c-c586a033cfd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:30.309137 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3719979-ab92-4946-867c-c586a033cfd0 None admin] PATH_INFO: `/v3/projects/e646d320aa5a4f78bfccaf941cf46478/users/9dc9ea8f89c14a2a8e3272f750647dfd/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:30.311104 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3719979-ab92-4946-867c-c586a033cfd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=e646d320aa5a4f78bfccaf941cf46478, user_id=9dc9ea8f89c14a2a8e3272f750647dfd, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:30.312562 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3719979-ab92-4946-867c-c586a033cfd0 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:30.325606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 381/764] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 27 04:09:30 2022] PUT /identity/v3/projects/e646d320aa5a4f78bfccaf941cf46478/users/9dc9ea8f89c14a2a8e3272f750647dfd/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:09:30.326903 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fabaf-fba5-4df9-845a-75d7a415b1f5 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:30.327192 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fabaf-fba5-4df9-845a-75d7a415b1f5 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:30.327465 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fabaf-fba5-4df9-845a-75d7a415b1f5 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:30.329992 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 382/765] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:30.330449 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11fabaf-fba5-4df9-845a-75d7a415b1f5 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:30.367064 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:30.426447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b902c-8222-425a-8ad8-798e6c217cfc None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:30.456339 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11fabaf-fba5-4df9-845a-75d7a415b1f5 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:30.479398 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bbbc71-1851-41f0-b04b-1edd5aa52bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:30.480366 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bbbc71-1851-41f0-b04b-1edd5aa52bbb None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:30.480833 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bbbc71-1851-41f0-b04b-1edd5aa52bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:30.481362 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bbbc71-1851-41f0-b04b-1edd5aa52bbb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:30.482380 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bbbc71-1851-41f0-b04b-1edd5aa52bbb None admin] RBAC: Adding query filter params (interface=public, service_id=7e59dbb554e84c87b4c5f7202e385156, region_id=RegionOne) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:30.482898 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bbbc71-1851-41f0-b04b-1edd5aa52bbb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:30.484467 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bbbc71-1851-41f0-b04b-1edd5aa52bbb None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:30.494802 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 384/766] 10.222.0.46 () {56 vars in 1251 bytes} [Wed Jul 27 04:09:30 2022] GET /identity/v3/endpoints?service_id=7e59dbb554e84c87b4c5f7202e385156&interface=public®ion_id=RegionOne => generated 194 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:30.496693 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904f630-5d6c-4eb3-96ba-0d6c4e74db07 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:30.497318 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904f630-5d6c-4eb3-96ba-0d6c4e74db07 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:30.497839 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904f630-5d6c-4eb3-96ba-0d6c4e74db07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:30.498202 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035d1ce-f603-4414-8dda-4459f85b7053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:30.498824 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035d1ce-f603-4414-8dda-4459f85b7053 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:30.499102 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035d1ce-f603-4414-8dda-4459f85b7053 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:30.499463 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035d1ce-f603-4414-8dda-4459f85b7053 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:30.500275 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1035d1ce-f603-4414-8dda-4459f85b7053 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:30.500788 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1035d1ce-f603-4414-8dda-4459f85b7053 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:30.502182 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1035d1ce-f603-4414-8dda-4459f85b7053 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:30.518833 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-a904f630-5d6c-4eb3-96ba-0d6c4e74db07 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:30.520352 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a904f630-5d6c-4eb3-96ba-0d6c4e74db07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:30.521141 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 383/767] 10.222.0.46 () {56 vars in 1149 bytes} [Wed Jul 27 04:09:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:30.522073 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035d1ce-f603-4414-8dda-4459f85b7053 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:30.557586 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:30.615987 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 385/768] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:09:30.617477 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904f630-5d6c-4eb3-96ba-0d6c4e74db07 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:30.642093 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:30.656269 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035d1ce-f603-4414-8dda-4459f85b7053 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:30.695367 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb9beb-e6de-47c4-976b-3628e50349ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:30.696124 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb9beb-e6de-47c4-976b-3628e50349ed None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:30.696597 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb9beb-e6de-47c4-976b-3628e50349ed None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:30.696887 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb9beb-e6de-47c4-976b-3628e50349ed None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:30.697568 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb9beb-e6de-47c4-976b-3628e50349ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:30.698979 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb9beb-e6de-47c4-976b-3628e50349ed None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:30.707289 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 384/769] 10.222.0.46 () {56 vars in 1087 bytes} [Wed Jul 27 04:09:30 2022] GET /identity/v3/services => generated 772 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:30.708249 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb9beb-e6de-47c4-976b-3628e50349ed None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:30.746210 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904f630-5d6c-4eb3-96ba-0d6c4e74db07 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:30.788379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b76f8f1-a94e-47b0-828b-f230e1da94a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:30.789411 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76f8f1-a94e-47b0-828b-f230e1da94a6 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:30.789911 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76f8f1-a94e-47b0-828b-f230e1da94a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:30.790441 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76f8f1-a94e-47b0-828b-f230e1da94a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:30.834168 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:30.878573 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b76f8f1-a94e-47b0-828b-f230e1da94a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:30.880389 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b76f8f1-a94e-47b0-828b-f230e1da94a6 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:30.961149 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb9beb-e6de-47c4-976b-3628e50349ed None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:31.005346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 386/770] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:09:30 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 04:09:31.011396 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b76f8f1-a94e-47b0-828b-f230e1da94a6 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:31.022187 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c061c-cb39-42d9-8f6a-4d052edd3e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:31.023230 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c061c-cb39-42d9-8f6a-4d052edd3e73 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:31.023715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c061c-cb39-42d9-8f6a-4d052edd3e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:31.024191 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c061c-cb39-42d9-8f6a-4d052edd3e73 None admin] PATH_INFO: `/v3/services/network` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:31.026360 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c061c-cb39-42d9-8f6a-4d052edd3e73 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:31.028081 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c061c-cb39-42d9-8f6a-4d052edd3e73 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:31.039075 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-5d5c061c-cb39-42d9-8f6a-4d052edd3e73 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 04:09:31.039075 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:31.039075 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 04:09:31.039075 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 04:09:31.039075 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 04:09:31.039075 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 04:09:31.039075 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 04:09:31.039075 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:31.039075 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 04:09:31.039075 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:31.039075 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 04:09:31.039075 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:31.039075 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:31.039075 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:31.039075 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:31.039075 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:31.039075 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:31.040095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:31.040095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:31.040095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:31.040095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:31.040095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:31.040095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:31.040095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 04:09:31.040095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 04:09:31.040095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 04:09:31.040095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 04:09:31.040095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:31.040095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:31.040095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:31.040095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:31.040095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:31.040095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:31.040910 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:31.040910 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:31.040910 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:31.040910 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:31.040910 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:31.040910 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:31.040910 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:31.040910 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:31.040910 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:31.040910 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:31.040910 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 04:09:31.040910 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:31.040910 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 04:09:31.040910 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:31.043925 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 385/771] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:30 2022] GET /identity/v3/services/network => generated 88 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:31.051920 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c061c-cb39-42d9-8f6a-4d052edd3e73 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:31.080837 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:31.093829 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:31.175899 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c061c-cb39-42d9-8f6a-4d052edd3e73 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:31.176400 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b76f8f1-a94e-47b0-828b-f230e1da94a6 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:31.239868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb6f6d-436d-445f-859e-3c22de6c99b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:31.241745 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb6f6d-436d-445f-859e-3c22de6c99b9 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:31.242549 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb6f6d-436d-445f-859e-3c22de6c99b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:31.243559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb6f6d-436d-445f-859e-3c22de6c99b9 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:31.244394 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c496b0-85bc-415a-81dd-ae5ed36d35c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:31.245644 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eb6f6d-436d-445f-859e-3c22de6c99b9 None admin] RBAC: Adding query filter params (name=network) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:31.246509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c496b0-85bc-415a-81dd-ae5ed36d35c3 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:31.247016 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eb6f6d-436d-445f-859e-3c22de6c99b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:31.248578 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eb6f6d-436d-445f-859e-3c22de6c99b9 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:31.249084 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c496b0-85bc-415a-81dd-ae5ed36d35c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:31.249555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c496b0-85bc-415a-81dd-ae5ed36d35c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:31.261705 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 386/772] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Jul 27 04:09:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:31.267866 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb6f6d-436d-445f-859e-3c22de6c99b9 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:31.270727 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c496b0-85bc-415a-81dd-ae5ed36d35c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:31.277514 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c496b0-85bc-415a-81dd-ae5ed36d35c3 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:31.301777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-e1c496b0-85bc-415a-81dd-ae5ed36d35c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:31.301777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:31.301777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:31.301777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:31.301777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:31.301777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:31.301777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:31.301777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:31.301777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:31.301777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:31.301777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:31.301777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:31.301777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:31.301777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:31.301777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:31.302728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:31.302728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:31.302728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:31.302728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:31.302728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:31.302728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:31.302728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:31.302728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:31.302728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:31.302728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:31.302728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:31.302728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:31.302728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:31.302728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:31.302728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:31.302728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:31.302728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:31.303616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:31.303616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:31.303616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:31.303616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:31.310334 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 387/773] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:31.311854 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c496b0-85bc-415a-81dd-ae5ed36d35c3 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:31.378848 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:31.398080 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb6f6d-436d-445f-859e-3c22de6c99b9 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:31.464754 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06564c3e-0463-40ac-8a8a-41c939235317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:31.465391 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06564c3e-0463-40ac-8a8a-41c939235317 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:31.465669 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06564c3e-0463-40ac-8a8a-41c939235317 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:31.465938 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06564c3e-0463-40ac-8a8a-41c939235317 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:31.469674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06564c3e-0463-40ac-8a8a-41c939235317 None admin] RBAC: Adding query filter params (type=network) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:31.469674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06564c3e-0463-40ac-8a8a-41c939235317 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:31.469674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06564c3e-0463-40ac-8a8a-41c939235317 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:31.477704 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 387/774] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Jul 27 04:09:31 2022] GET /identity/v3/services?type=network => generated 350 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:09:31.482461 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06564c3e-0463-40ac-8a8a-41c939235317 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:31.515015 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c496b0-85bc-415a-81dd-ae5ed36d35c3 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:31.559424 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:31.575975 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7736a4d-c634-4db9-a685-a0b1f904ad8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:31.576736 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7736a4d-c634-4db9-a685-a0b1f904ad8a None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:31.577084 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7736a4d-c634-4db9-a685-a0b1f904ad8a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:31.577523 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7736a4d-c634-4db9-a685-a0b1f904ad8a None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:31.578323 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7736a4d-c634-4db9-a685-a0b1f904ad8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:31.579241 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7736a4d-c634-4db9-a685-a0b1f904ad8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:31.580689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7736a4d-c634-4db9-a685-a0b1f904ad8a None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:31.593659 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 388/775] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:09:31.595144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e3920-ecaf-4142-8359-bc6070d02a83 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:31.595430 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e3920-ecaf-4142-8359-bc6070d02a83 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:31.595706 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e3920-ecaf-4142-8359-bc6070d02a83 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:31.600166 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06564c3e-0463-40ac-8a8a-41c939235317 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:31.601332 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 389/776] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:31.602232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e3920-ecaf-4142-8359-bc6070d02a83 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:31.635473 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea80562d-0510-410d-a5d1-4f0e295f5a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:31.636110 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80562d-0510-410d-a5d1-4f0e295f5a6e None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:31.636390 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80562d-0510-410d-a5d1-4f0e295f5a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:31.636663 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80562d-0510-410d-a5d1-4f0e295f5a6e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:31.637435 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea80562d-0510-410d-a5d1-4f0e295f5a6e None admin] RBAC: Adding query filter params (interface=public, service_id=372591e914434844853ffa7bc79b999b, region_id=RegionOne) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:31.637745 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea80562d-0510-410d-a5d1-4f0e295f5a6e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:31.639156 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea80562d-0510-410d-a5d1-4f0e295f5a6e None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:31.649430 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 388/777] 10.222.0.46 () {56 vars in 1251 bytes} [Wed Jul 27 04:09:31 2022] GET /identity/v3/endpoints?service_id=372591e914434844853ffa7bc79b999b&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:09:31.650803 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7870b3a-bc19-496a-b201-76faf0b63e72 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:31.651089 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7870b3a-bc19-496a-b201-76faf0b63e72 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:31.651364 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7870b3a-bc19-496a-b201-76faf0b63e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:31.682000 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-b7870b3a-bc19-496a-b201-76faf0b63e72 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:31.682000 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b7870b3a-bc19-496a-b201-76faf0b63e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:31.686802 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:31.727535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e3920-ecaf-4142-8359-bc6070d02a83 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:31.759977 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 389/778] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:09:31.761557 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7870b3a-bc19-496a-b201-76faf0b63e72 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:31.770915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83487dfd-9129-4f2c-b2f7-7240f0428d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:31.771609 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83487dfd-9129-4f2c-b2f7-7240f0428d2f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:31.772432 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83487dfd-9129-4f2c-b2f7-7240f0428d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:31.772717 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83487dfd-9129-4f2c-b2f7-7240f0428d2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:31.782011 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83487dfd-9129-4f2c-b2f7-7240f0428d2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:31.784199 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83487dfd-9129-4f2c-b2f7-7240f0428d2f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:31.798382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-83487dfd-9129-4f2c-b2f7-7240f0428d2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:31.798382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:31.798382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:31.798382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:31.798382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:31.798382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:31.798382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:31.798382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:31.798382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:31.798382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:31.798382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:31.798382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:31.798382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:09:31.798382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:09:31.798382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:09:31.799275 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:09:31.799275 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:31.799275 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:31.799275 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:31.799275 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:31.799275 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:31.799275 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:31.799275 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:31.799275 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:31.799275 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:31.799275 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:31.799275 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:31.799275 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:31.799275 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:31.799275 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:31.799275 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:31.799275 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:31.800362 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:09:31.800362 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:09:31.800362 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:09:31.800362 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:09:31.800362 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:09:31.800362 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:09:31.800362 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:31.800362 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:31.800362 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 390/779] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:31.802226 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83487dfd-9129-4f2c-b2f7-7240f0428d2f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:31.854965 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:31.893350 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7870b3a-bc19-496a-b201-76faf0b63e72 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:31.931083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83487dfd-9129-4f2c-b2f7-7240f0428d2f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:31.934003 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469d2d0-89a8-4c5b-bbb5-912422171f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:31.934688 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469d2d0-89a8-4c5b-bbb5-912422171f67 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:31.934970 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469d2d0-89a8-4c5b-bbb5-912422171f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:31.935242 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469d2d0-89a8-4c5b-bbb5-912422171f67 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:31.935955 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0469d2d0-89a8-4c5b-bbb5-912422171f67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:31.937325 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0469d2d0-89a8-4c5b-bbb5-912422171f67 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:31.943730 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 390/780] 10.222.0.46 () {56 vars in 1087 bytes} [Wed Jul 27 04:09:31 2022] GET /identity/v3/services => generated 772 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:31.944593 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469d2d0-89a8-4c5b-bbb5-912422171f67 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:31.948978 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:31.975086 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd1535-5a6b-4c2e-9e8e-6274a3b0c4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:31.976505 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd1535-5a6b-4c2e-9e8e-6274a3b0c4c2 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:31.976788 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd1535-5a6b-4c2e-9e8e-6274a3b0c4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:31.977066 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd1535-5a6b-4c2e-9e8e-6274a3b0c4c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:31.992066 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd1535-5a6b-4c2e-9e8e-6274a3b0c4c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:09:31.999188 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-1ddd1535-5a6b-4c2e-9e8e-6274a3b0c4c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:09:31.999188 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:31.999188 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:31.999188 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:31.999188 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:31.999188 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:31.999188 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:31.999188 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:31.999188 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:31.999188 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:31.999188 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:31.999188 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:31.999188 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:09:31.999188 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:09:31.999188 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:09:31.999188 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:09:32.001289 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:09:32.001289 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:09:32.001289 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:09:32.001289 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:09:32.001289 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:32.001289 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:32.001289 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:32.001289 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:32.001289 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:32.001289 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:32.001289 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:32.001289 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:32.001289 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:32.001289 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:32.001289 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:32.001289 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:32.002370 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:32.002370 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:32.002370 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:32.002370 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:32.002370 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:09:32.002370 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:09:32.002370 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:09:32.002370 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:09:32.002370 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:09:32.002370 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:09:32.002370 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:09:32.002370 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:32.002370 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 391/781] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:09:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:32.009492 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd1535-5a6b-4c2e-9e8e-6274a3b0c4c2 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:32.074000 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469d2d0-89a8-4c5b-bbb5-912422171f67 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:32.110840 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e0221-1346-44f5-8050-2ecf863b9799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:32.111926 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e0221-1346-44f5-8050-2ecf863b9799 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:32.112784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e0221-1346-44f5-8050-2ecf863b9799 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:32.113067 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e0221-1346-44f5-8050-2ecf863b9799 None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:32.114030 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5e0221-1346-44f5-8050-2ecf863b9799 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:32.114496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5e0221-1346-44f5-8050-2ecf863b9799 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:32.116726 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5e0221-1346-44f5-8050-2ecf863b9799 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:32.130908 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:32.134243 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd1535-5a6b-4c2e-9e8e-6274a3b0c4c2 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:32.135046 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 391/782] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:09:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:32.143030 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e0221-1346-44f5-8050-2ecf863b9799 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:32.205621 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cbbe18-58a3-47e1-8546-b1eeed5e90d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:32.207354 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cbbe18-58a3-47e1-8546-b1eeed5e90d9 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:32.207977 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cbbe18-58a3-47e1-8546-b1eeed5e90d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:32.209836 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cbbe18-58a3-47e1-8546-b1eeed5e90d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:32.211374 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cbbe18-58a3-47e1-8546-b1eeed5e90d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:32.211688 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cbbe18-58a3-47e1-8546-b1eeed5e90d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:32.213089 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cbbe18-58a3-47e1-8546-b1eeed5e90d9 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:32.229914 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 392/783] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:09:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:09:32.235223 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cbbe18-58a3-47e1-8546-b1eeed5e90d9 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:32.267664 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:32.321752 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e0221-1346-44f5-8050-2ecf863b9799 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:32.365312 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52846605-9441-4b2c-8d0f-7b084546c974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:32.366090 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52846605-9441-4b2c-8d0f-7b084546c974 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:32.366471 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52846605-9441-4b2c-8d0f-7b084546c974 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:32.366949 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52846605-9441-4b2c-8d0f-7b084546c974 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:32.369476 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52846605-9441-4b2c-8d0f-7b084546c974 None admin] RBAC: Adding query filter params (role.id=6521132834a64b029695bbdd5eb251d2, scope.project.id=e646d320aa5a4f78bfccaf941cf46478, user.id=8ec07ee1efba4e8089b94f24be338f86) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:32.369858 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52846605-9441-4b2c-8d0f-7b084546c974 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:32.371444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52846605-9441-4b2c-8d0f-7b084546c974 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:32.374113 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cbbe18-58a3-47e1-8546-b1eeed5e90d9 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:32.379020 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 392/784] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:09:32 2022] GET /identity/v3/role_assignments?user.id=8ec07ee1efba4e8089b94f24be338f86&scope.project.id=e646d320aa5a4f78bfccaf941cf46478&role.id=6521132834a64b029695bbdd5eb251d2 => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:32.423318 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50b5fe-615a-492b-9937-57000a992db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:32.424350 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50b5fe-615a-492b-9937-57000a992db5 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:32.424820 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50b5fe-615a-492b-9937-57000a992db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:32.425284 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50b5fe-615a-492b-9937-57000a992db5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:32.446641 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50b5fe-615a-492b-9937-57000a992db5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:32.448464 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50b5fe-615a-492b-9937-57000a992db5 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:32.470570 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-ec50b5fe-615a-492b-9937-57000a992db5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:32.470570 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:32.470570 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:32.470570 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:32.470570 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:32.470570 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:32.470570 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:32.470570 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:32.470570 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:32.470570 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:32.470570 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:32.470570 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:32.470570 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:32.470570 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:32.470570 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:32.471558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:32.471558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:32.471558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:32.471558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:32.471558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:32.471558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:32.471558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:32.471558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:32.471558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:32.471558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:32.471558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:32.471558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:32.471558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:32.471558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:32.471558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:32.471558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:32.471558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:32.472465 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:32.472465 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:32.472465 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:32.472465 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:32.476558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 393/785] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:32.479135 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52846605-9441-4b2c-8d0f-7b084546c974 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:32.648614 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52846605-9441-4b2c-8d0f-7b084546c974 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:32.683618 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:32.694718 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32351c19-c115-4706-ae3c-f8d304150013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:32.695576 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32351c19-c115-4706-ae3c-f8d304150013 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:32.696008 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32351c19-c115-4706-ae3c-f8d304150013 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:32.696455 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32351c19-c115-4706-ae3c-f8d304150013 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:32.697304 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32351c19-c115-4706-ae3c-f8d304150013 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:32.697756 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32351c19-c115-4706-ae3c-f8d304150013 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:32.699308 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32351c19-c115-4706-ae3c-f8d304150013 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:32.718601 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 393/786] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:32.724323 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50b5fe-615a-492b-9937-57000a992db5 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:32.746481 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:32.811555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3691ca-7d36-413e-8d9e-f11facd4b71d None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:32.812121 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3691ca-7d36-413e-8d9e-f11facd4b71d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:32.812549 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3691ca-7d36-413e-8d9e-f11facd4b71d None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:32.815656 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 394/787] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:32.835434 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91404b-fbc0-4ddd-a50d-82d7c5fd3ffd None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:32.835944 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91404b-fbc0-4ddd-a50d-82d7c5fd3ffd None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:32.836372 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91404b-fbc0-4ddd-a50d-82d7c5fd3ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:32.857428 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50b5fe-615a-492b-9937-57000a992db5 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:32.883014 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-9e91404b-fbc0-4ddd-a50d-82d7c5fd3ffd None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:32.884502 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9e91404b-fbc0-4ddd-a50d-82d7c5fd3ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:32.895316 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedeea39-277c-4a79-b0f6-3cb5ca63558a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:32.895981 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedeea39-277c-4a79-b0f6-3cb5ca63558a None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:32.896269 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedeea39-277c-4a79-b0f6-3cb5ca63558a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:32.896547 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedeea39-277c-4a79-b0f6-3cb5ca63558a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:32.910079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedeea39-277c-4a79-b0f6-3cb5ca63558a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:32.910079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedeea39-277c-4a79-b0f6-3cb5ca63558a None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:32.919097 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-dedeea39-277c-4a79-b0f6-3cb5ca63558a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 04:09:32.919097 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:32.919097 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:32.919097 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:32.919097 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:32.919097 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:32.919097 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:32.919097 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:32.919097 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:32.919097 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:32.919097 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:32.919097 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:32.919097 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:09:32.919097 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:09:32.919097 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:09:32.919097 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:09:32.920251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:32.920251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:32.920251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:09:32.920251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:32.920251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:09:32.920251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:32.920251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:32.920251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:32.920251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:32.920251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:32.920251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:32.920251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:32.920251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:32.920251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:32.920251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:32.920251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:32.920251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:32.920251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:32.921421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:32.921421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:32.921421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:09:32.921421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:09:32.921421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:09:32.921421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:09:32.921421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:09:32.921421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:09:32.921421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 04:09:32.921421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:32.929787 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 394/788] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:09:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:32.929787 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedeea39-277c-4a79-b0f6-3cb5ca63558a None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:33.031435 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 395/789] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:09:33.053587 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e91404b-fbc0-4ddd-a50d-82d7c5fd3ffd None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:33.090018 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:33.147115 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedeea39-277c-4a79-b0f6-3cb5ca63558a None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:33.220968 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e91404b-fbc0-4ddd-a50d-82d7c5fd3ffd None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:33.230670 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9c88e-1544-40e0-a95b-1a33c26974d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:33.231314 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9c88e-1544-40e0-a95b-1a33c26974d3 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:33.231599 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9c88e-1544-40e0-a95b-1a33c26974d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:33.231876 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9c88e-1544-40e0-a95b-1a33c26974d3 None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:33.232687 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9c88e-1544-40e0-a95b-1a33c26974d3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:33.232997 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9c88e-1544-40e0-a95b-1a33c26974d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:33.234463 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9c88e-1544-40e0-a95b-1a33c26974d3 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:33.265633 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 395/790] 10.222.0.46 () {56 vars in 1135 bytes} [Wed Jul 27 04:09:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:33.270359 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:33.273595 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9c88e-1544-40e0-a95b-1a33c26974d3 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:33.291484 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfeb5b1-e784-4264-a938-396dbbac0876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:33.292142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfeb5b1-e784-4264-a938-396dbbac0876 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:33.292424 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfeb5b1-e784-4264-a938-396dbbac0876 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:33.292702 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfeb5b1-e784-4264-a938-396dbbac0876 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:33.315074 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfeb5b1-e784-4264-a938-396dbbac0876 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:09:33.320715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-6cfeb5b1-e784-4264-a938-396dbbac0876 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:09:33.320715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:33.320715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:33.320715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:33.320715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:33.320715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:33.320715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:33.320715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:33.320715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:33.320715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:33.320715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:33.320715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:33.320715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:09:33.320715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:09:33.320715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:09:33.320715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:09:33.321645 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:09:33.321645 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:09:33.321645 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:09:33.321645 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:09:33.321645 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:33.321645 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:33.321645 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:33.321645 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:33.321645 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:33.321645 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:33.321645 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:33.321645 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:33.321645 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:33.321645 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:33.321645 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:33.321645 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:33.322554 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:33.322554 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:33.322554 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:33.322554 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:33.322554 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:09:33.322554 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:09:33.322554 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:09:33.322554 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:09:33.322554 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:09:33.322554 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:09:33.322554 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:09:33.322554 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:33.330104 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 396/791] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:09:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:09:33.331732 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0295cd-02cf-4c74-827e-853c92e1e1a9 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:33.332203 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0295cd-02cf-4c74-827e-853c92e1e1a9 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:33.332619 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0295cd-02cf-4c74-827e-853c92e1e1a9 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:33.338186 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 397/792] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:33.340276 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0295cd-02cf-4c74-827e-853c92e1e1a9 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:33.418290 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9c88e-1544-40e0-a95b-1a33c26974d3 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:33.459436 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:33.466270 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da654dd-f5cf-4bbe-bd6a-384217dfa5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:33.467088 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da654dd-f5cf-4bbe-bd6a-384217dfa5ed None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:33.467372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da654dd-f5cf-4bbe-bd6a-384217dfa5ed None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:33.467648 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da654dd-f5cf-4bbe-bd6a-384217dfa5ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:33.484115 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da654dd-f5cf-4bbe-bd6a-384217dfa5ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:33.486105 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da654dd-f5cf-4bbe-bd6a-384217dfa5ed None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:33.488406 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0295cd-02cf-4c74-827e-853c92e1e1a9 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:33.504564 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-7da654dd-f5cf-4bbe-bd6a-384217dfa5ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:33.504564 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:33.504564 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:33.504564 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:33.504564 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:33.504564 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:33.504564 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:33.504564 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:33.504564 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:33.504564 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:33.504564 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:33.504564 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:33.504564 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:33.504564 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:33.504564 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:33.505694 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:33.505694 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:33.505694 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:33.505694 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:33.505694 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:33.505694 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:33.505694 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:33.505694 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:33.505694 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:33.505694 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:33.505694 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:33.505694 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:33.505694 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:33.505694 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:33.505694 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:33.505694 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:33.505694 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:33.506942 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:33.506942 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:33.506942 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:33.506942 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:33.510110 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 396/793] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:09:33.512465 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35c8c8-2191-4110-b31d-f2596507efa4 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:33.512756 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35c8c8-2191-4110-b31d-f2596507efa4 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:33.513035 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35c8c8-2191-4110-b31d-f2596507efa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:33.530944 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-7d35c8c8-2191-4110-b31d-f2596507efa4 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:33.532317 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7d35c8c8-2191-4110-b31d-f2596507efa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:33.535899 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd16a9-5a00-41bd-8894-08591c0bacf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:33.536760 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd16a9-5a00-41bd-8894-08591c0bacf1 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:33.537243 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd16a9-5a00-41bd-8894-08591c0bacf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:33.537744 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd16a9-5a00-41bd-8894-08591c0bacf1 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:33.538767 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cd16a9-5a00-41bd-8894-08591c0bacf1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:33.539317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cd16a9-5a00-41bd-8894-08591c0bacf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:33.540979 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cd16a9-5a00-41bd-8894-08591c0bacf1 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:33.550864 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 398/794] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:09:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:33.552082 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd16a9-5a00-41bd-8894-08591c0bacf1 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:33.597096 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:33.610327 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 397/795] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:09:33.610327 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d35c8c8-2191-4110-b31d-f2596507efa4 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:33.685400 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd16a9-5a00-41bd-8894-08591c0bacf1 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:33.726692 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7afaccd-cacc-44d0-8791-5293c58f54c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:33.727949 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afaccd-cacc-44d0-8791-5293c58f54c1 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:33.728453 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afaccd-cacc-44d0-8791-5293c58f54c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:33.728931 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afaccd-cacc-44d0-8791-5293c58f54c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:33.729183 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d35c8c8-2191-4110-b31d-f2596507efa4 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:33.730151 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7afaccd-cacc-44d0-8791-5293c58f54c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:33.730804 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7afaccd-cacc-44d0-8791-5293c58f54c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:33.733298 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7afaccd-cacc-44d0-8791-5293c58f54c1 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:33.746130 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:33.753542 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 399/796] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:33.754697 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7afaccd-cacc-44d0-8791-5293c58f54c1 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:33.771732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19854ea-45d4-43d5-ab8a-6fef7cd4093a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:33.772388 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19854ea-45d4-43d5-ab8a-6fef7cd4093a None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:33.772733 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19854ea-45d4-43d5-ab8a-6fef7cd4093a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:33.773013 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19854ea-45d4-43d5-ab8a-6fef7cd4093a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:33.785167 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19854ea-45d4-43d5-ab8a-6fef7cd4093a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:33.786822 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19854ea-45d4-43d5-ab8a-6fef7cd4093a None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:33.805048 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-f19854ea-45d4-43d5-ab8a-6fef7cd4093a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:33.805048 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:33.805048 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:33.805048 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:33.805048 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:33.805048 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:33.805048 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:33.805048 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:33.805048 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:33.805048 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:33.805048 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:33.805048 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:33.805048 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:33.805048 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:33.805048 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:33.806057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:33.806057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:33.806057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:33.806057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:33.806057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:33.806057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:33.806057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:33.806057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:33.806057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:33.806057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:33.806057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:33.806057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:33.806057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:33.806057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:33.806057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:33.806057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:33.806057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:33.806961 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:33.806961 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:33.806961 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:33.806961 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:33.806961 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 398/797] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:33.807679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19854ea-45d4-43d5-ab8a-6fef7cd4093a None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:33.862592 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:33.865853 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7afaccd-cacc-44d0-8791-5293c58f54c1 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:33.902436 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15882f49-92a7-402e-8bd0-a8de2d3968f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:33.903835 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15882f49-92a7-402e-8bd0-a8de2d3968f1 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:33.904357 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15882f49-92a7-402e-8bd0-a8de2d3968f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:33.904862 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15882f49-92a7-402e-8bd0-a8de2d3968f1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:33.905712 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15882f49-92a7-402e-8bd0-a8de2d3968f1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:33.908445 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15882f49-92a7-402e-8bd0-a8de2d3968f1 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:33.919377 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19854ea-45d4-43d5-ab8a-6fef7cd4093a None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:33.920631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-15882f49-92a7-402e-8bd0-a8de2d3968f1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 04:09:33.920631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:33.920631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 04:09:33.920631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 04:09:33.920631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 04:09:33.920631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 04:09:33.920631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 04:09:33.920631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:33.920631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 04:09:33.920631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:33.920631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 04:09:33.920631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:33.920631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:33.920631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:33.920631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:33.920631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:33.921508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:33.921508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:33.921508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:33.921508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:33.921508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:33.921508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:33.921508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:33.921508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 04:09:33.921508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 04:09:33.921508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 04:09:33.921508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 04:09:33.921508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:33.921508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:33.921508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:33.921508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:33.921508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:33.922481 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:33.922481 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:33.922481 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:33.922481 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:33.922481 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:33.922481 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:33.922481 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:33.922481 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:33.922481 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:33.922481 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:33.922481 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:33.922481 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 04:09:33.922481 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:33.922481 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 04:09:33.922481 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:33.930415 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 400/798] 10.222.0.46 () {56 vars in 1129 bytes} [Wed Jul 27 04:09:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:33.930415 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15882f49-92a7-402e-8bd0-a8de2d3968f1 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:33.964113 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c508ed6-5c87-419d-9091-5386701872d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:33.964891 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c508ed6-5c87-419d-9091-5386701872d6 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:33.965163 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c508ed6-5c87-419d-9091-5386701872d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:33.965453 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c508ed6-5c87-419d-9091-5386701872d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:33.973916 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c508ed6-5c87-419d-9091-5386701872d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:33.975457 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c508ed6-5c87-419d-9091-5386701872d6 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:33.988086 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-3c508ed6-5c87-419d-9091-5386701872d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:33.988086 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:33.988086 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:33.988086 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:33.988086 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:33.988086 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:33.988086 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:33.988086 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:33.988086 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:33.988086 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:33.988086 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:33.988086 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:33.988086 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:09:33.988086 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:09:33.988086 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:09:33.989040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:09:33.989040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:33.989040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:33.989040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:33.989040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:33.989040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:33.989040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:33.989040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:33.989040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:33.989040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:33.989040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:33.989040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:33.989040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:33.989040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:33.989040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:33.989040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:33.989040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:33.989911 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:09:33.989911 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:09:33.989911 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:09:33.989911 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:09:33.989911 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:09:33.989911 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:09:33.989911 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:33.989911 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:33.994997 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 399/799] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:33.996100 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c508ed6-5c87-419d-9091-5386701872d6 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:34.042479 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:34.058658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15882f49-92a7-402e-8bd0-a8de2d3968f1 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:34.095290 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d9359-62eb-4046-a043-a23f240c9068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:34.096435 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d9359-62eb-4046-a043-a23f240c9068 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:34.096958 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d9359-62eb-4046-a043-a23f240c9068 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:34.097446 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d9359-62eb-4046-a043-a23f240c9068 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:34.098383 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d9359-62eb-4046-a043-a23f240c9068 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:34.098911 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d9359-62eb-4046-a043-a23f240c9068 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:34.100491 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d9359-62eb-4046-a043-a23f240c9068 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:34.114929 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 401/800] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:34.116071 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d9359-62eb-4046-a043-a23f240c9068 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:34.120798 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c508ed6-5c87-419d-9091-5386701872d6 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:34.144784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:34.171753 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730fd6f-f28a-4470-a323-4f964089608c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:34.173753 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730fd6f-f28a-4470-a323-4f964089608c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:34.174084 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730fd6f-f28a-4470-a323-4f964089608c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:34.174474 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730fd6f-f28a-4470-a323-4f964089608c None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:34.175195 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6730fd6f-f28a-4470-a323-4f964089608c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:34.175570 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6730fd6f-f28a-4470-a323-4f964089608c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:34.177546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6730fd6f-f28a-4470-a323-4f964089608c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:34.189455 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 400/801] 10.222.0.46 () {56 vars in 1124 bytes} [Wed Jul 27 04:09:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:34.189455 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730fd6f-f28a-4470-a323-4f964089608c None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:34.258879 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d9359-62eb-4046-a043-a23f240c9068 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:34.301217 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344771b7-b8d4-492c-8a5e-5bbecc9993b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:34.302685 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344771b7-b8d4-492c-8a5e-5bbecc9993b1 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:34.303194 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344771b7-b8d4-492c-8a5e-5bbecc9993b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:34.303984 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344771b7-b8d4-492c-8a5e-5bbecc9993b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:34.306129 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344771b7-b8d4-492c-8a5e-5bbecc9993b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:34.306678 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344771b7-b8d4-492c-8a5e-5bbecc9993b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:34.308442 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344771b7-b8d4-492c-8a5e-5bbecc9993b1 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:34.320288 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:34.336653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 402/802] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:09:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:34.337256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730fd6f-f28a-4470-a323-4f964089608c None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:34.337639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344771b7-b8d4-492c-8a5e-5bbecc9993b1 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:34.379545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b7062-edf1-41b0-9459-b15938476d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:34.380275 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b7062-edf1-41b0-9459-b15938476d0f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:34.380550 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b7062-edf1-41b0-9459-b15938476d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:34.380958 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b7062-edf1-41b0-9459-b15938476d0f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:34.389311 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4b7062-edf1-41b0-9459-b15938476d0f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:34.390794 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4b7062-edf1-41b0-9459-b15938476d0f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:34.408990 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-2e4b7062-edf1-41b0-9459-b15938476d0f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 04:09:34.408990 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:34.408990 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:34.408990 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:34.408990 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:34.408990 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:34.408990 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:34.408990 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:34.408990 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:34.408990 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:34.408990 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:34.408990 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:34.408990 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:09:34.408990 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:09:34.408990 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:09:34.408990 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:09:34.410047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:34.410047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:34.410047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:09:34.410047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:34.410047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:09:34.410047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:34.410047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:34.410047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:34.410047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:34.410047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:34.410047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:34.410047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:34.410047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:34.410047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:34.410047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:34.410047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:34.410047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:34.410047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:34.410986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:34.410986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:34.410986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:09:34.410986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:09:34.410986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:09:34.410986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:09:34.410986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:09:34.410986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:09:34.410986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 04:09:34.410986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:34.410986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 401/803] 10.222.0.46 () {56 vars in 1093 bytes} [Wed Jul 27 04:09:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:34.412077 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764a0fe-cde7-4877-b174-2d3829931cc2 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:34.412284 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764a0fe-cde7-4877-b174-2d3829931cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:34.412829 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764a0fe-cde7-4877-b174-2d3829931cc2 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:34.418654 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 402/804] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:34.418654 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764a0fe-cde7-4877-b174-2d3829931cc2 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:34.468977 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:34.492973 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344771b7-b8d4-492c-8a5e-5bbecc9993b1 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:34.540228 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e4876-6f02-4e7e-8a8b-5f4c9ee08915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:34.540867 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e4876-6f02-4e7e-8a8b-5f4c9ee08915 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:34.541148 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e4876-6f02-4e7e-8a8b-5f4c9ee08915 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:34.541426 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e4876-6f02-4e7e-8a8b-5f4c9ee08915 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:34.542161 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e4876-6f02-4e7e-8a8b-5f4c9ee08915 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:34.542468 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e4876-6f02-4e7e-8a8b-5f4c9ee08915 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:34.543962 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e4876-6f02-4e7e-8a8b-5f4c9ee08915 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:34.554832 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 403/805] 10.222.0.46 () {56 vars in 1124 bytes} [Wed Jul 27 04:09:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:34.555543 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e4876-6f02-4e7e-8a8b-5f4c9ee08915 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:34.559333 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764a0fe-cde7-4877-b174-2d3829931cc2 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:34.601375 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc3a9d-dafd-42ee-8be3-1425b5b86dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:34.602007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc3a9d-dafd-42ee-8be3-1425b5b86dcd None admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:34.602298 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc3a9d-dafd-42ee-8be3-1425b5b86dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:34.603240 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc3a9d-dafd-42ee-8be3-1425b5b86dcd None admin] PATH_INFO: `/v3/projects/e646d320aa5a4f78bfccaf941cf46478/users/57b5fa7517034a39930ce7c9d12e1875/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:34.604619 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfc3a9d-dafd-42ee-8be3-1425b5b86dcd None admin] RBAC: Authorizing `identity:create_grant(project_id=e646d320aa5a4f78bfccaf941cf46478, user_id=57b5fa7517034a39930ce7c9d12e1875, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:34.606097 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfc3a9d-dafd-42ee-8be3-1425b5b86dcd None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:34.626883 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 403/806] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 27 04:09:34 2022] PUT /identity/v3/projects/e646d320aa5a4f78bfccaf941cf46478/users/57b5fa7517034a39930ce7c9d12e1875/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:09:34.628575 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b69b23-ec9a-420d-b46e-2f1c98b160c6 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:34.629068 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b69b23-ec9a-420d-b46e-2f1c98b160c6 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:34.630109 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b69b23-ec9a-420d-b46e-2f1c98b160c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:34.658930 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-21b69b23-ec9a-420d-b46e-2f1c98b160c6 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:34.660257 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-21b69b23-ec9a-420d-b46e-2f1c98b160c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:34.666408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:34.702137 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e4876-6f02-4e7e-8a8b-5f4c9ee08915 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:34.759520 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb07665-302b-44f2-a444-515e15f70a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:34.760191 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb07665-302b-44f2-a444-515e15f70a3a None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:34.760478 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb07665-302b-44f2-a444-515e15f70a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:34.760756 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb07665-302b-44f2-a444-515e15f70a3a None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:34.762340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb07665-302b-44f2-a444-515e15f70a3a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:34.762665 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb07665-302b-44f2-a444-515e15f70a3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:34.765198 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb07665-302b-44f2-a444-515e15f70a3a None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:34.770652 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 404/807] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:09:34.771940 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b69b23-ec9a-420d-b46e-2f1c98b160c6 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:34.793163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 404/808] 10.222.0.46 () {56 vars in 1133 bytes} [Wed Jul 27 04:09:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:34.801591 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb07665-302b-44f2-a444-515e15f70a3a None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:34.812398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:34.936761 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb07665-302b-44f2-a444-515e15f70a3a None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:34.943084 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b69b23-ec9a-420d-b46e-2f1c98b160c6 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:34.982623 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420647ad-c09e-48e1-8f46-3b44991906ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:34.983255 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420647ad-c09e-48e1-8f46-3b44991906ca None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:34.983621 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420647ad-c09e-48e1-8f46-3b44991906ca None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:34.983821 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420647ad-c09e-48e1-8f46-3b44991906ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:34.989485 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5c28e-9b2a-4d54-a23c-7ecc21908828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:34.990817 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5c28e-9b2a-4d54-a23c-7ecc21908828 None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:34.991498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5c28e-9b2a-4d54-a23c-7ecc21908828 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:34.992157 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5c28e-9b2a-4d54-a23c-7ecc21908828 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:34.993718 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a5c28e-9b2a-4d54-a23c-7ecc21908828 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:34.995584 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a5c28e-9b2a-4d54-a23c-7ecc21908828 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:35.008664 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:35.034006 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 405/809] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:09:34 2022] POST /identity/v3/endpoints => generated 346 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:09:35.034006 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5c28e-9b2a-4d54-a23c-7ecc21908828 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:35.102715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420647ad-c09e-48e1-8f46-3b44991906ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:35.104739 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420647ad-c09e-48e1-8f46-3b44991906ca None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:35.116564 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:35.155354 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5c28e-9b2a-4d54-a23c-7ecc21908828 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:35.197597 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2010b-7d00-4ff7-8d5b-6dd60a58c9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:35.198348 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2010b-7d00-4ff7-8d5b-6dd60a58c9bd None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:35.198627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2010b-7d00-4ff7-8d5b-6dd60a58c9bd None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:35.198897 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2010b-7d00-4ff7-8d5b-6dd60a58c9bd None admin] PATH_INFO: `/v3/services/network` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:35.199581 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2010b-7d00-4ff7-8d5b-6dd60a58c9bd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:35.201085 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2010b-7d00-4ff7-8d5b-6dd60a58c9bd None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:35.212927 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-8dc2010b-7d00-4ff7-8d5b-6dd60a58c9bd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 04:09:35.212927 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:35.212927 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 04:09:35.212927 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 04:09:35.212927 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 04:09:35.212927 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 04:09:35.212927 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 04:09:35.212927 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:35.212927 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 04:09:35.212927 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:35.212927 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 04:09:35.212927 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:35.212927 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:35.212927 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:35.212927 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:35.212927 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:35.212927 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:35.214126 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:35.214126 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:35.214126 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:35.214126 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:35.214126 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:35.214126 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:35.214126 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 04:09:35.214126 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 04:09:35.214126 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 04:09:35.214126 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 04:09:35.214126 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:35.214126 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:35.214126 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:35.214126 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:35.214126 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:35.214126 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:35.215138 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:35.215138 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:35.215138 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:35.215138 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:35.215138 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:35.215138 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:35.215138 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:35.215138 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:35.215138 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:35.215138 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:35.215138 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 04:09:35.215138 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:35.215138 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 04:09:35.215138 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:35.215138 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 406/810] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:35 2022] GET /identity/v3/services/network => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:35.215929 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c47e7-4b00-4f9f-83f4-b6fae29ba12b None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:35.215929 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c47e7-4b00-4f9f-83f4-b6fae29ba12b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:35.216252 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c47e7-4b00-4f9f-83f4-b6fae29ba12b None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:35.217804 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 407/811] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:35.224191 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c47e7-4b00-4f9f-83f4-b6fae29ba12b None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:35.258483 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 405/812] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:09:34 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 04:09:35.259778 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7aae4b-457a-4157-ae63-32e7c0abded3 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:35.260069 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7aae4b-457a-4157-ae63-32e7c0abded3 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:35.260352 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7aae4b-457a-4157-ae63-32e7c0abded3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:35.281716 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-3e7aae4b-457a-4157-ae63-32e7c0abded3 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:35.283672 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3e7aae4b-457a-4157-ae63-32e7c0abded3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:35.297406 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:35.356214 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c47e7-4b00-4f9f-83f4-b6fae29ba12b None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:35.374004 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 406/813] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:09:35.378412 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7aae4b-457a-4157-ae63-32e7c0abded3 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:35.418112 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca5bc1-e509-4840-9340-35b614f90a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:35.419309 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca5bc1-e509-4840-9340-35b614f90a84 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:35.420647 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca5bc1-e509-4840-9340-35b614f90a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:35.421423 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca5bc1-e509-4840-9340-35b614f90a84 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:35.422958 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca5bc1-e509-4840-9340-35b614f90a84 None admin] RBAC: Adding query filter params (name=network) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:35.423835 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca5bc1-e509-4840-9340-35b614f90a84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:35.426225 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca5bc1-e509-4840-9340-35b614f90a84 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:35.432650 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:35.434741 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 408/814] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Jul 27 04:09:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:35.440105 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca5bc1-e509-4840-9340-35b614f90a84 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:35.512156 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7aae4b-457a-4157-ae63-32e7c0abded3 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:35.549601 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b919c8-1123-4d4b-98b5-ae6530835872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:35.550366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b919c8-1123-4d4b-98b5-ae6530835872 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:35.550738 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b919c8-1123-4d4b-98b5-ae6530835872 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:35.551089 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b919c8-1123-4d4b-98b5-ae6530835872 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:35.562225 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b919c8-1123-4d4b-98b5-ae6530835872 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:35.564278 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b919c8-1123-4d4b-98b5-ae6530835872 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:35.582839 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca5bc1-e509-4840-9340-35b614f90a84 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:35.587897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-40b919c8-1123-4d4b-98b5-ae6530835872 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:35.587897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:35.587897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:35.587897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:35.587897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:35.587897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:35.587897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:35.587897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:35.587897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:35.587897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:35.587897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:35.587897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:35.587897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:35.587897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:35.587897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:35.588780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:35.588780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:35.588780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:35.588780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:35.588780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:35.588780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:35.588780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:35.588780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:35.588780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:35.588780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:35.588780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:35.588780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:35.588780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:35.588780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:35.588780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:35.588780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:35.588780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:35.589695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:35.589695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:35.589695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:35.589695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:35.592261 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 407/815] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:35.592261 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b919c8-1123-4d4b-98b5-ae6530835872 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:35.610831 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:35.633724 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44b433c-885e-4ad8-94dd-91f3b9e25fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:35.634424 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b433c-885e-4ad8-94dd-91f3b9e25fd0 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:35.634704 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b433c-885e-4ad8-94dd-91f3b9e25fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:35.634975 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b433c-885e-4ad8-94dd-91f3b9e25fd0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:35.645761 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44b433c-885e-4ad8-94dd-91f3b9e25fd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:09:35.651240 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-d44b433c-885e-4ad8-94dd-91f3b9e25fd0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:35.651240 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:35.651240 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:35.651240 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:35.651240 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:35.651240 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:35.651240 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:35.651240 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:35.651240 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:35.651240 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:35.651240 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:35.651240 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:35.651240 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:09:35.651240 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:09:35.651240 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:09:35.651240 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:09:35.652172 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:09:35.652172 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:09:35.652172 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:09:35.652172 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:09:35.652172 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:35.652172 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:35.652172 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:35.652172 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:35.652172 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:35.652172 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:35.652172 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:35.652172 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:35.652172 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:35.652172 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:35.652172 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:35.652172 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:35.653013 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:35.653013 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:35.653013 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:35.653013 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:35.653013 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:09:35.653013 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:09:35.653013 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:09:35.653013 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:09:35.653013 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:09:35.653013 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:09:35.653013 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:35.653013 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:35.657292 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 409/816] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 27 04:09:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:35.658190 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44b433c-885e-4ad8-94dd-91f3b9e25fd0 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:35.708719 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:35.722026 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b919c8-1123-4d4b-98b5-ae6530835872 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:35.757603 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ea436-b500-46d9-8ed7-71eb96d93bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:35.758267 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ea436-b500-46d9-8ed7-71eb96d93bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:35.758602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ea436-b500-46d9-8ed7-71eb96d93bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:35.758875 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ea436-b500-46d9-8ed7-71eb96d93bc1 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:35.759611 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6ea436-b500-46d9-8ed7-71eb96d93bc1 None admin] RBAC: Adding query filter params (type=network) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:35.759917 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6ea436-b500-46d9-8ed7-71eb96d93bc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:35.761291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6ea436-b500-46d9-8ed7-71eb96d93bc1 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:35.767407 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 408/817] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Jul 27 04:09:35 2022] GET /identity/v3/services?type=network => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:35.768221 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ea436-b500-46d9-8ed7-71eb96d93bc1 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:35.786164 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44b433c-885e-4ad8-94dd-91f3b9e25fd0 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:35.833073 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c1f2a-c956-4734-b9e4-80f76f02160a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:35.834653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c1f2a-c956-4734-b9e4-80f76f02160a None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:35.835131 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c1f2a-c956-4734-b9e4-80f76f02160a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:35.835623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c1f2a-c956-4734-b9e4-80f76f02160a None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:35.836525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c1f2a-c956-4734-b9e4-80f76f02160a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:35.837179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c1f2a-c956-4734-b9e4-80f76f02160a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:35.839155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c1f2a-c956-4734-b9e4-80f76f02160a None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:35.858025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 410/818] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:35.858025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c1f2a-c956-4734-b9e4-80f76f02160a None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:35.886497 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ea436-b500-46d9-8ed7-71eb96d93bc1 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:35.909814 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:35.923524 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456f6a0-9150-4a0f-b1bf-a749394dd114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:35.924364 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456f6a0-9150-4a0f-b1bf-a749394dd114 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:35.924795 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456f6a0-9150-4a0f-b1bf-a749394dd114 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:35.925293 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456f6a0-9150-4a0f-b1bf-a749394dd114 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:35.926193 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c456f6a0-9150-4a0f-b1bf-a749394dd114 None admin] RBAC: Adding query filter params (name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:35.926715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c456f6a0-9150-4a0f-b1bf-a749394dd114 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:35.928275 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c456f6a0-9150-4a0f-b1bf-a749394dd114 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:35.937063 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 409/819] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 27 04:09:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:35.938866 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456f6a0-9150-4a0f-b1bf-a749394dd114 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:35.955075 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:35.976689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c1f2a-c956-4734-b9e4-80f76f02160a None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:36.015005 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb5d4a-9589-4a43-ad39-7bd2afce4f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:36.015855 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb5d4a-9589-4a43-ad39-7bd2afce4f86 None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:36.016597 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb5d4a-9589-4a43-ad39-7bd2afce4f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:36.017122 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb5d4a-9589-4a43-ad39-7bd2afce4f86 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:36.018092 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb5d4a-9589-4a43-ad39-7bd2afce4f86 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:36.019773 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb5d4a-9589-4a43-ad39-7bd2afce4f86 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:36.043163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456f6a0-9150-4a0f-b1bf-a749394dd114 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:36.043378 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 411/820] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:09:35 2022] POST /identity/v3/endpoints => generated 327 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:09:36.044737 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb5d4a-9589-4a43-ad39-7bd2afce4f86 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:36.111596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ec469-ecad-4247-82dc-76915171a891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:36.112289 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ec469-ecad-4247-82dc-76915171a891 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:36.112572 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ec469-ecad-4247-82dc-76915171a891 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:36.112847 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ec469-ecad-4247-82dc-76915171a891 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:36.126185 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ec469-ecad-4247-82dc-76915171a891 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:36.134492 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ec469-ecad-4247-82dc-76915171a891 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:36.155789 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-cc0ec469-ecad-4247-82dc-76915171a891 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:36.155789 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:36.155789 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:36.155789 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:36.155789 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:36.155789 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:36.155789 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:36.155789 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:36.155789 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:36.155789 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:36.155789 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:36.155789 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:36.155789 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:09:36.155789 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:09:36.155789 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:09:36.156902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:09:36.156902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:36.156902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:36.156902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:36.156902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:36.156902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:36.156902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:36.156902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:36.156902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:36.156902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:36.156902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:36.156902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:36.156902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:36.156902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:36.156902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:36.156902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:36.156902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:36.157894 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:09:36.157894 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:09:36.157894 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:09:36.157894 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:09:36.157894 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:09:36.157894 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:09:36.157894 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:36.157894 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:36.157894 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 410/821] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:36.171819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:36.175042 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ec469-ecad-4247-82dc-76915171a891 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:36.180199 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb5d4a-9589-4a43-ad39-7bd2afce4f86 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:36.228786 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b21ada-9059-4686-9be4-a22ee5ca299f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:36.229442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b21ada-9059-4686-9be4-a22ee5ca299f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:36.229719 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b21ada-9059-4686-9be4-a22ee5ca299f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:36.230055 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b21ada-9059-4686-9be4-a22ee5ca299f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:36.242679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b21ada-9059-4686-9be4-a22ee5ca299f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:36.244130 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b21ada-9059-4686-9be4-a22ee5ca299f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:36.261032 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-e8b21ada-9059-4686-9be4-a22ee5ca299f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:36.261032 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:36.261032 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:36.261032 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:36.261032 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:36.261032 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:36.261032 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:36.261032 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:36.261032 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:36.261032 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:36.261032 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:36.261032 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:36.261032 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:36.261032 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:36.261032 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:36.262002 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:36.262002 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:36.262002 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:36.262002 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:36.262002 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:36.262002 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:36.262002 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:36.262002 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:36.262002 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:36.262002 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:36.262002 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:36.262002 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:36.262002 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:36.262002 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:36.262002 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:36.262002 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:36.262002 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:36.262941 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:36.262941 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:36.262941 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:36.262941 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:36.263656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 412/822] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:36.269198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b21ada-9059-4686-9be4-a22ee5ca299f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:36.290222 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:36.336968 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ec469-ecad-4247-82dc-76915171a891 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:36.396379 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995ed2b-d0a2-41d4-a17f-42721e7412e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:36.397193 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995ed2b-d0a2-41d4-a17f-42721e7412e4 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:36.397458 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995ed2b-d0a2-41d4-a17f-42721e7412e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:36.397731 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995ed2b-d0a2-41d4-a17f-42721e7412e4 None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:36.398536 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0995ed2b-d0a2-41d4-a17f-42721e7412e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:36.398849 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0995ed2b-d0a2-41d4-a17f-42721e7412e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:36.400601 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0995ed2b-d0a2-41d4-a17f-42721e7412e4 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:36.417225 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b21ada-9059-4686-9be4-a22ee5ca299f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:36.425071 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 411/823] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:09:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:36.426742 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995ed2b-d0a2-41d4-a17f-42721e7412e4 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:36.462277 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f33a9d-487c-4b5d-9aef-81c1224a2a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:36.462909 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f33a9d-487c-4b5d-9aef-81c1224a2a5d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:36.463194 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f33a9d-487c-4b5d-9aef-81c1224a2a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:36.470000 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f33a9d-487c-4b5d-9aef-81c1224a2a5d None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:36.470000 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f33a9d-487c-4b5d-9aef-81c1224a2a5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:36.470000 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f33a9d-487c-4b5d-9aef-81c1224a2a5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:36.470000 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f33a9d-487c-4b5d-9aef-81c1224a2a5d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:36.483264 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 413/824] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:36.489445 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f33a9d-487c-4b5d-9aef-81c1224a2a5d None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:36.508206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:36.539074 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995ed2b-d0a2-41d4-a17f-42721e7412e4 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:36.569227 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:36.575441 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc4a553-0347-4be6-ba9b-9a58bbdd7a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:36.576065 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4a553-0347-4be6-ba9b-9a58bbdd7a6f None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:36.576833 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4a553-0347-4be6-ba9b-9a58bbdd7a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:36.576833 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4a553-0347-4be6-ba9b-9a58bbdd7a6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:36.577499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc4a553-0347-4be6-ba9b-9a58bbdd7a6f None admin] RBAC: Adding query filter params (role.id=cf771a8e94ee46149eb081347fea6d67, scope.project.id=e646d320aa5a4f78bfccaf941cf46478, user.id=9dc9ea8f89c14a2a8e3272f750647dfd) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:36.577808 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc4a553-0347-4be6-ba9b-9a58bbdd7a6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:36.580107 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc4a553-0347-4be6-ba9b-9a58bbdd7a6f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:36.592953 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 412/825] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:09:36 2022] GET /identity/v3/role_assignments?user.id=9dc9ea8f89c14a2a8e3272f750647dfd&scope.project.id=e646d320aa5a4f78bfccaf941cf46478&role.id=cf771a8e94ee46149eb081347fea6d67 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:36.619208 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f33a9d-487c-4b5d-9aef-81c1224a2a5d None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:36.661330 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a7ac51-37f0-4be9-9468-8f1e68ad4c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:36.662291 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7ac51-37f0-4be9-9468-8f1e68ad4c96 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:36.662722 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7ac51-37f0-4be9-9468-8f1e68ad4c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:36.663149 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7ac51-37f0-4be9-9468-8f1e68ad4c96 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:36.671973 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a7ac51-37f0-4be9-9468-8f1e68ad4c96 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:36.674565 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a7ac51-37f0-4be9-9468-8f1e68ad4c96 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:36.688671 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-b3a7ac51-37f0-4be9-9468-8f1e68ad4c96 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 04:09:36.688671 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:36.688671 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:36.688671 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:36.688671 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:36.688671 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:36.688671 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:36.688671 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:36.688671 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:36.688671 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:36.688671 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:36.688671 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:36.688671 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:09:36.688671 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:09:36.688671 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:09:36.688671 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:09:36.690278 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:36.690278 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:36.690278 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:09:36.690278 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:36.690278 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:09:36.690278 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:36.690278 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:36.690278 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:36.690278 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:36.690278 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:36.690278 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:36.690278 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:36.690278 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:36.690278 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:36.690278 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:36.690278 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:36.690278 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:36.690278 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:36.691451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:36.691451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:36.691451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:09:36.691451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:09:36.691451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:09:36.691451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:09:36.691451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:09:36.691451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:09:36.691451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 04:09:36.691451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:36.693835 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 414/826] 10.222.0.46 () {56 vars in 1117 bytes} [Wed Jul 27 04:09:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:36.706098 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc4a553-0347-4be6-ba9b-9a58bbdd7a6f None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:36.812468 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0254f19-6984-41cf-a5e4-259686c53827 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:36.813597 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0254f19-6984-41cf-a5e4-259686c53827 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:36.813851 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc4a553-0347-4be6-ba9b-9a58bbdd7a6f None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:36.813985 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0254f19-6984-41cf-a5e4-259686c53827 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:36.815498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 415/827] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 04:09:36.821197 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bc4cf-ff9b-4b61-b926-c02ecce4114c None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:36.821490 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bc4cf-ff9b-4b61-b926-c02ecce4114c None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:36.821775 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bc4cf-ff9b-4b61-b926-c02ecce4114c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:36.842991 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-be1bc4cf-ff9b-4b61-b926-c02ecce4114c None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:36.844854 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-be1bc4cf-ff9b-4b61-b926-c02ecce4114c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:36.853083 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2cb79-a4ab-4be6-b950-af6e0a24ac99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:36.854031 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2cb79-a4ab-4be6-b950-af6e0a24ac99 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:36.854532 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2cb79-a4ab-4be6-b950-af6e0a24ac99 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:36.855094 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2cb79-a4ab-4be6-b950-af6e0a24ac99 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:36.856094 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a2cb79-a4ab-4be6-b950-af6e0a24ac99 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:36.856715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a2cb79-a4ab-4be6-b950-af6e0a24ac99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:36.858439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a2cb79-a4ab-4be6-b950-af6e0a24ac99 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:36.879505 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 413/828] 10.222.0.46 () {56 vars in 1149 bytes} [Wed Jul 27 04:09:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:36.884773 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2cb79-a4ab-4be6-b950-af6e0a24ac99 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:36.915294 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:36.925388 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 416/829] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:36 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:09:36.949675 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1bc4cf-ff9b-4b61-b926-c02ecce4114c None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:36.954596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:37.005484 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2cb79-a4ab-4be6-b950-af6e0a24ac99 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:37.050622 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c90f35-4af2-41fe-80f6-cbf590471db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:37.051272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c90f35-4af2-41fe-80f6-cbf590471db1 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:37.051893 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c90f35-4af2-41fe-80f6-cbf590471db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:37.052173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c90f35-4af2-41fe-80f6-cbf590471db1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:37.057630 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1bc4cf-ff9b-4b61-b926-c02ecce4114c None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:37.064788 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c90f35-4af2-41fe-80f6-cbf590471db1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:37.066689 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c90f35-4af2-41fe-80f6-cbf590471db1 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:37.086456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-a4c90f35-4af2-41fe-80f6-cbf590471db1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:37.086456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:37.086456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:37.086456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:37.086456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:37.086456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:37.086456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:37.086456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:37.086456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:37.086456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:37.086456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:37.086456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:37.086456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:37.086456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:37.086456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:37.087430 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:37.087430 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:37.087430 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:37.087430 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:37.087430 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:37.087430 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:37.087430 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:37.087430 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:37.087430 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:37.087430 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:37.087430 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:37.087430 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:37.087430 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:37.087430 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:37.087430 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:37.087430 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:37.087430 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:37.088377 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:37.088377 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:37.088377 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:37.088377 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:37.088377 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 414/830] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:09:37.093151 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c90f35-4af2-41fe-80f6-cbf590471db1 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:37.107622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a6249-3249-4ad1-a273-70f8027cda64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:37.109026 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a6249-3249-4ad1-a273-70f8027cda64 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:37.109862 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a6249-3249-4ad1-a273-70f8027cda64 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:37.110407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a6249-3249-4ad1-a273-70f8027cda64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:37.126167 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a6249-3249-4ad1-a273-70f8027cda64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:09:37.133689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-b65a6249-3249-4ad1-a273-70f8027cda64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:09:37.133689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:37.133689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:37.133689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:37.133689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:37.133689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:37.133689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:37.133689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:37.133689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:37.133689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:37.133689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:37.133689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:37.133689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:09:37.133689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:09:37.133689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:09:37.133689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:09:37.134731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:09:37.134731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:09:37.134731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:09:37.134731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:09:37.134731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:37.134731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:37.134731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:37.134731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:37.134731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:37.134731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:37.134731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:37.134731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:37.134731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:37.134731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:37.134731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:37.134731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:37.135600 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:37.135600 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:37.135600 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:37.135600 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:37.135600 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:09:37.135600 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:09:37.135600 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:09:37.135600 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:09:37.135600 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:09:37.135600 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:09:37.135600 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:09:37.135600 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:37.144388 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 417/831] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:09:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:37.145267 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a6249-3249-4ad1-a273-70f8027cda64 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:37.211663 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c90f35-4af2-41fe-80f6-cbf590471db1 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:37.214705 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:37.220241 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:37.251987 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38509b78-b601-46c0-86a6-22428224b561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:37.253098 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38509b78-b601-46c0-86a6-22428224b561 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:37.253624 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38509b78-b601-46c0-86a6-22428224b561 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:37.254180 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38509b78-b601-46c0-86a6-22428224b561 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:37.255125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38509b78-b601-46c0-86a6-22428224b561 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:37.255629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38509b78-b601-46c0-86a6-22428224b561 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:37.257286 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38509b78-b601-46c0-86a6-22428224b561 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:37.275161 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a6249-3249-4ad1-a273-70f8027cda64 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:37.275773 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 415/832] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:09:37.277021 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38509b78-b601-46c0-86a6-22428224b561 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:37.311858 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3b6b5f-758e-4a33-a170-44fa9e95767b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:37.312499 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b6b5f-758e-4a33-a170-44fa9e95767b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:37.313854 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b6b5f-758e-4a33-a170-44fa9e95767b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:37.314159 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b6b5f-758e-4a33-a170-44fa9e95767b None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:37.314889 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3b6b5f-758e-4a33-a170-44fa9e95767b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:37.315192 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3b6b5f-758e-4a33-a170-44fa9e95767b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:37.316734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3b6b5f-758e-4a33-a170-44fa9e95767b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:37.331516 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 418/833] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:09:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:09:37.333381 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc719135-630a-4c25-bcf0-15d0b927fbed None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:37.333922 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc719135-630a-4c25-bcf0-15d0b927fbed None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:37.334246 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc719135-630a-4c25-bcf0-15d0b927fbed None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:37.336230 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 419/834] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:37.337356 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc719135-630a-4c25-bcf0-15d0b927fbed None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:37.390701 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38509b78-b601-46c0-86a6-22428224b561 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:37.428309 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629ef6f-45fc-49d2-8e55-c1429b647715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:37.429046 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629ef6f-45fc-49d2-8e55-c1429b647715 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:37.429400 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629ef6f-45fc-49d2-8e55-c1429b647715 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:37.429680 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629ef6f-45fc-49d2-8e55-c1429b647715 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:37.437004 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629ef6f-45fc-49d2-8e55-c1429b647715 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:37.438932 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629ef6f-45fc-49d2-8e55-c1429b647715 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:37.450285 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-8629ef6f-45fc-49d2-8e55-c1429b647715 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:37.450285 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:37.450285 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:37.450285 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:37.450285 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:37.450285 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:37.450285 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:37.450285 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:37.450285 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:37.450285 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:37.450285 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:37.450285 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:37.450285 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:09:37.450285 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:09:37.450285 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:09:37.451209 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:09:37.451209 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:37.451209 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:37.451209 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:37.451209 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:37.451209 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:37.451209 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:37.451209 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:37.451209 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:37.451209 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:37.451209 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:37.451209 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:37.451209 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:37.451209 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:37.451209 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:37.451209 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:37.451209 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:37.452776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:09:37.452776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:09:37.452776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:09:37.452776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:09:37.452776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:09:37.452776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:09:37.452776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:37.452776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:37.452776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 416/835] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:37.453267 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90452dce-934f-4a2f-bb56-2331a15a9985 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:37.453267 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90452dce-934f-4a2f-bb56-2331a15a9985 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:37.453704 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90452dce-934f-4a2f-bb56-2331a15a9985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:37.466593 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc719135-630a-4c25-bcf0-15d0b927fbed None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:37.474229 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-90452dce-934f-4a2f-bb56-2331a15a9985 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:37.475841 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-90452dce-934f-4a2f-bb56-2331a15a9985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:37.489717 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:37.496037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:37.505234 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5893441-0725-47ca-9884-1b4f9e3321f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:37.505875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5893441-0725-47ca-9884-1b4f9e3321f8 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:37.506182 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5893441-0725-47ca-9884-1b4f9e3321f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:37.506456 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5893441-0725-47ca-9884-1b4f9e3321f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:37.517467 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5893441-0725-47ca-9884-1b4f9e3321f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:37.518940 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5893441-0725-47ca-9884-1b4f9e3321f8 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:37.533915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-c5893441-0725-47ca-9884-1b4f9e3321f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:37.533915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:37.533915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:37.533915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:37.533915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:37.533915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:37.533915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:37.533915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:37.533915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:37.533915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:37.533915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:37.533915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:37.533915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:37.533915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:37.533915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:37.534969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:37.534969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:37.534969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:37.534969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:37.534969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:37.534969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:37.534969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:37.534969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:37.534969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:37.534969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:37.534969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:37.534969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:37.534969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:37.534969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:37.534969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:37.534969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:37.534969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:37.535884 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:37.535884 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:37.535884 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:37.535884 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:37.535884 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 420/836] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:09:37.536490 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5893441-0725-47ca-9884-1b4f9e3321f8 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:37.558017 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 417/837] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:37 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:09:37.558017 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90452dce-934f-4a2f-bb56-2331a15a9985 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:37.647399 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5893441-0725-47ca-9884-1b4f9e3321f8 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:37.671223 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90452dce-934f-4a2f-bb56-2331a15a9985 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:37.683339 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b705e-f498-4f70-9c84-0095beeb68d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:37.683968 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b705e-f498-4f70-9c84-0095beeb68d3 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:37.684251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b705e-f498-4f70-9c84-0095beeb68d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:37.684528 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b705e-f498-4f70-9c84-0095beeb68d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:37.685284 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b705e-f498-4f70-9c84-0095beeb68d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:37.685589 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b705e-f498-4f70-9c84-0095beeb68d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:37.687050 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b705e-f498-4f70-9c84-0095beeb68d3 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:37.699537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 421/838] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:09:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:37.700335 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b705e-f498-4f70-9c84-0095beeb68d3 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:37.715964 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34fc0b1-7853-4ab6-8520-1e62e20a1aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:37.717069 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fc0b1-7853-4ab6-8520-1e62e20a1aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:37.717880 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fc0b1-7853-4ab6-8520-1e62e20a1aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:37.718366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fc0b1-7853-4ab6-8520-1e62e20a1aa7 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:37.720020 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34fc0b1-7853-4ab6-8520-1e62e20a1aa7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:37.720506 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34fc0b1-7853-4ab6-8520-1e62e20a1aa7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:37.722492 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34fc0b1-7853-4ab6-8520-1e62e20a1aa7 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:37.739334 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 418/839] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:37.739334 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34fc0b1-7853-4ab6-8520-1e62e20a1aa7 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:37.744353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:37.752282 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:37.822218 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b705e-f498-4f70-9c84-0095beeb68d3 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:37.848747 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34fc0b1-7853-4ab6-8520-1e62e20a1aa7 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:37.857822 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f9278-6d38-4aa3-a022-baf267ffad56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:37.858507 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f9278-6d38-4aa3-a022-baf267ffad56 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:37.858790 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f9278-6d38-4aa3-a022-baf267ffad56 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:37.859068 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f9278-6d38-4aa3-a022-baf267ffad56 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:37.860267 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f9278-6d38-4aa3-a022-baf267ffad56 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:37.861756 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f9278-6d38-4aa3-a022-baf267ffad56 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:37.870898 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-5d3f9278-6d38-4aa3-a022-baf267ffad56 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 04:09:37.870898 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:37.870898 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 04:09:37.870898 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 04:09:37.870898 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 04:09:37.870898 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 04:09:37.870898 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 04:09:37.870898 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:37.870898 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 04:09:37.870898 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:37.870898 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 04:09:37.870898 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:37.870898 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:37.870898 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:37.870898 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:37.870898 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:37.870898 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:37.871875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:37.871875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:37.871875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:37.871875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:37.871875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:37.871875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:37.871875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 04:09:37.871875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 04:09:37.871875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 04:09:37.871875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 04:09:37.871875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:37.871875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:37.871875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:37.871875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:37.871875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:37.871875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:37.872690 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:37.872690 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:37.872690 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:37.872690 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:37.872690 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:37.872690 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:37.872690 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:37.872690 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:37.872690 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:37.872690 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:37.872690 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 04:09:37.872690 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:37.872690 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 04:09:37.872690 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:37.872690 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 422/840] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 27 04:09:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:37.873553 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f9278-6d38-4aa3-a022-baf267ffad56 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:37.891838 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d828a-6326-4cc3-ae6f-e5ed4543798a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:37.892824 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d828a-6326-4cc3-ae6f-e5ed4543798a None admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:37.893271 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d828a-6326-4cc3-ae6f-e5ed4543798a None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:37.893686 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d828a-6326-4cc3-ae6f-e5ed4543798a None admin] PATH_INFO: `/v3/projects/e646d320aa5a4f78bfccaf941cf46478/users/8ec07ee1efba4e8089b94f24be338f86/roles/6521132834a64b029695bbdd5eb251d2` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:37.909567 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d828a-6326-4cc3-ae6f-e5ed4543798a None admin] RBAC: Authorizing `identity:create_grant(project_id=e646d320aa5a4f78bfccaf941cf46478, user_id=8ec07ee1efba4e8089b94f24be338f86, role_id=6521132834a64b029695bbdd5eb251d2)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:37.911214 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d828a-6326-4cc3-ae6f-e5ed4543798a None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:37.926619 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 419/841] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 27 04:09:37 2022] PUT /identity/v3/projects/e646d320aa5a4f78bfccaf941cf46478/users/8ec07ee1efba4e8089b94f24be338f86/roles/6521132834a64b029695bbdd5eb251d2 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:09:37.927860 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d828a-6326-4cc3-ae6f-e5ed4543798a None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:37.979935 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:37.987117 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f9278-6d38-4aa3-a022-baf267ffad56 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:38.027213 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197f328-fea8-46e3-8646-0c3e7650f413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:38.027921 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197f328-fea8-46e3-8646-0c3e7650f413 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:38.028155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197f328-fea8-46e3-8646-0c3e7650f413 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:38.028434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197f328-fea8-46e3-8646-0c3e7650f413 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:38.032848 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:38.038420 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f197f328-fea8-46e3-8646-0c3e7650f413 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:38.039913 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f197f328-fea8-46e3-8646-0c3e7650f413 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:38.055617 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-f197f328-fea8-46e3-8646-0c3e7650f413 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 04:09:38.055617 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:38.055617 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:38.055617 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:38.055617 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:38.055617 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:38.055617 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:38.055617 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:38.055617 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:38.055617 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:38.055617 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:38.055617 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:38.055617 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:09:38.055617 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:09:38.055617 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:09:38.055617 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:09:38.056627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:38.056627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:38.056627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:09:38.056627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:38.056627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:09:38.056627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:38.056627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:38.056627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:38.056627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:38.056627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:38.056627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:38.056627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:38.056627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:38.056627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:38.056627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:38.056627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:38.056627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:38.056627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:38.062513 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:38.062513 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:38.062513 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:09:38.062513 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:09:38.062513 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:09:38.062513 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:09:38.062513 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:09:38.062513 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:09:38.062513 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 04:09:38.062513 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:38.062513 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 423/842] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:09:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:38.063461 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197f328-fea8-46e3-8646-0c3e7650f413 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:38.089800 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d828a-6326-4cc3-ae6f-e5ed4543798a None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:38.139964 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b67508-b620-47d9-ae38-f95206c3d833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:38.141385 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b67508-b620-47d9-ae38-f95206c3d833 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:38.142056 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b67508-b620-47d9-ae38-f95206c3d833 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:38.142353 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b67508-b620-47d9-ae38-f95206c3d833 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:38.143063 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b67508-b620-47d9-ae38-f95206c3d833 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:38.143374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b67508-b620-47d9-ae38-f95206c3d833 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:38.145639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b67508-b620-47d9-ae38-f95206c3d833 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:38.157699 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 420/843] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 27 04:09:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:38.159067 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b67508-b620-47d9-ae38-f95206c3d833 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:38.185919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197f328-fea8-46e3-8646-0c3e7650f413 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:38.227239 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a50cce-0b60-4f1b-8b38-6e4a0a9636ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:38.227873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a50cce-0b60-4f1b-8b38-6e4a0a9636ac None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:38.228156 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a50cce-0b60-4f1b-8b38-6e4a0a9636ac None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:38.228436 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a50cce-0b60-4f1b-8b38-6e4a0a9636ac None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:38.229239 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a50cce-0b60-4f1b-8b38-6e4a0a9636ac None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:38.229549 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a50cce-0b60-4f1b-8b38-6e4a0a9636ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:38.231052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a50cce-0b60-4f1b-8b38-6e4a0a9636ac None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:38.250365 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 424/844] 10.222.0.46 () {56 vars in 1135 bytes} [Wed Jul 27 04:09:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:38.255986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a50cce-0b60-4f1b-8b38-6e4a0a9636ac None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:38.272501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:38.299460 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b67508-b620-47d9-ae38-f95206c3d833 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:38.346617 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:38.352129 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b719f6a-99c8-44b5-be2c-5ad466f5561a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:38.352776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b719f6a-99c8-44b5-be2c-5ad466f5561a None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:38.353105 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b719f6a-99c8-44b5-be2c-5ad466f5561a None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:38.353388 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b719f6a-99c8-44b5-be2c-5ad466f5561a None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:38.354210 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b719f6a-99c8-44b5-be2c-5ad466f5561a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:38.354757 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b719f6a-99c8-44b5-be2c-5ad466f5561a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:38.356766 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b719f6a-99c8-44b5-be2c-5ad466f5561a None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:38.366067 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 421/845] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 27 04:09:38 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:09:38.373611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a50cce-0b60-4f1b-8b38-6e4a0a9636ac None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:38.418400 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c254ada6-429a-41ec-a8ff-f286f0875e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:38.419127 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254ada6-429a-41ec-a8ff-f286f0875e77 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:38.420202 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254ada6-429a-41ec-a8ff-f286f0875e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:38.420202 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254ada6-429a-41ec-a8ff-f286f0875e77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:38.502109 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c254ada6-429a-41ec-a8ff-f286f0875e77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:38.503533 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c254ada6-429a-41ec-a8ff-f286f0875e77 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:38.520715 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:38.612546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 425/846] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:09:38 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 27 04:09:38.618209 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b719f6a-99c8-44b5-be2c-5ad466f5561a None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:38.724200 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b719f6a-99c8-44b5-be2c-5ad466f5561a None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:38.754801 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155da11-4bd1-4fd8-9047-2454ace5fea9 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:38.755085 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155da11-4bd1-4fd8-9047-2454ace5fea9 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:38.755328 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155da11-4bd1-4fd8-9047-2454ace5fea9 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:38.756697 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 426/847] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 04:09:38.761490 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9eed8a-1476-4529-934e-08936aa0fdb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:38.762168 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9eed8a-1476-4529-934e-08936aa0fdb1 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:38.762450 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9eed8a-1476-4529-934e-08936aa0fdb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:38.762552 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d6c4c-ab9f-4e35-acce-ebccd1f70454 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:38.762722 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9eed8a-1476-4529-934e-08936aa0fdb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:38.762820 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d6c4c-ab9f-4e35-acce-ebccd1f70454 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:38.763077 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d6c4c-ab9f-4e35-acce-ebccd1f70454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:38.781891 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-0b3d6c4c-ab9f-4e35-acce-ebccd1f70454 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:38.782094 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9eed8a-1476-4529-934e-08936aa0fdb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:38.783192 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-0b3d6c4c-ab9f-4e35-acce-ebccd1f70454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:38.783563 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9eed8a-1476-4529-934e-08936aa0fdb1 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:38.800549 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-8c9eed8a-1476-4529-934e-08936aa0fdb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:38.800549 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:38.800549 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:38.800549 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:38.800549 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:38.800549 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:38.800549 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:38.800549 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:38.800549 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:38.800549 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:38.800549 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:38.800549 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:38.800549 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:38.800549 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:38.800549 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:38.801455 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:38.801455 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:38.801455 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:38.801455 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:38.801455 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:38.801455 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:38.801455 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:38.801455 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:38.801455 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:38.801455 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:38.801455 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:38.801455 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:38.801455 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:38.801455 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:38.801455 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:38.801455 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:38.801455 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:38.802403 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:38.802403 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:38.802403 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:38.802403 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:38.802403 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 422/848] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:38.807591 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9eed8a-1476-4529-934e-08936aa0fdb1 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:38.863274 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:38.874489 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 427/849] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:38 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 04:09:38.896530 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d6c4c-ab9f-4e35-acce-ebccd1f70454 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:38.933358 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9eed8a-1476-4529-934e-08936aa0fdb1 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:38.952041 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:38.974354 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6abb0-2b6c-4ecf-8530-fbab9f3bcb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:38.975422 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6abb0-2b6c-4ecf-8530-fbab9f3bcb1e None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:38.976004 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6abb0-2b6c-4ecf-8530-fbab9f3bcb1e None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:38.976507 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6abb0-2b6c-4ecf-8530-fbab9f3bcb1e None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:38.977467 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed6abb0-2b6c-4ecf-8530-fbab9f3bcb1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:38.977996 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed6abb0-2b6c-4ecf-8530-fbab9f3bcb1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:38.979689 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed6abb0-2b6c-4ecf-8530-fbab9f3bcb1e None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:38.997999 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 423/850] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:38.999163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6abb0-2b6c-4ecf-8530-fbab9f3bcb1e None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:39.015740 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d6c4c-ab9f-4e35-acce-ebccd1f70454 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:39.064737 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21e57d9-e303-4dd2-9a9f-a2bef26d5363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:39.065420 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e57d9-e303-4dd2-9a9f-a2bef26d5363 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:39.065939 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e57d9-e303-4dd2-9a9f-a2bef26d5363 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:39.066279 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e57d9-e303-4dd2-9a9f-a2bef26d5363 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:39.066923 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21e57d9-e303-4dd2-9a9f-a2bef26d5363 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:39.068450 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21e57d9-e303-4dd2-9a9f-a2bef26d5363 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:39.080115 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-b21e57d9-e303-4dd2-9a9f-a2bef26d5363 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 04:09:39.080115 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:39.080115 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 04:09:39.080115 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 04:09:39.080115 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 04:09:39.080115 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 04:09:39.080115 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 04:09:39.080115 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:39.080115 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 04:09:39.080115 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:39.080115 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 04:09:39.080115 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:39.080115 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:39.080115 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:39.080115 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:39.080115 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:39.081098 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:39.081098 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:39.081098 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:39.081098 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:39.081098 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:39.081098 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:39.081098 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:39.081098 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 04:09:39.081098 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 04:09:39.081098 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 04:09:39.081098 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 04:09:39.081098 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:39.081098 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:39.081098 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:39.081098 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:39.081098 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:39.081964 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:39.081964 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:39.081964 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:39.081964 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:39.081964 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:39.081964 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:39.081964 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:39.081964 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:39.081964 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:39.081964 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:39.081964 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:39.081964 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 04:09:39.081964 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:39.081964 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 04:09:39.081964 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:39.086052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 428/851] 10.222.0.46 () {56 vars in 1132 bytes} [Wed Jul 27 04:09:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:09:39.087526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21e57d9-e303-4dd2-9a9f-a2bef26d5363 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:39.122279 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6abb0-2b6c-4ecf-8530-fbab9f3bcb1e None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:39.125117 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:39.163351 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e0ec9-249a-455f-b616-1f2d85891550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:39.164612 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e0ec9-249a-455f-b616-1f2d85891550 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:39.165162 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e0ec9-249a-455f-b616-1f2d85891550 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:39.165641 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e0ec9-249a-455f-b616-1f2d85891550 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:39.173957 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781e0ec9-249a-455f-b616-1f2d85891550 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:39.175944 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781e0ec9-249a-455f-b616-1f2d85891550 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:39.190097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-781e0ec9-249a-455f-b616-1f2d85891550 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:39.190097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:39.190097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:39.190097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:39.190097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:39.190097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:39.190097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:39.190097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:39.190097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:39.190097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:39.190097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:39.190097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:39.190097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:09:39.190097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:09:39.190097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:09:39.191046 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:09:39.191046 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:39.191046 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:39.191046 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:39.191046 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:39.191046 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:39.191046 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:39.191046 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:39.191046 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:39.191046 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:39.191046 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:39.191046 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:39.191046 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:39.191046 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:39.191046 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:39.191046 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:39.191046 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:39.191959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:09:39.191959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:09:39.191959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:09:39.191959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:09:39.191959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:09:39.191959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:09:39.191959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:39.191959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:39.198365 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 424/852] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:39.200573 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e0ec9-249a-455f-b616-1f2d85891550 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:39.213110 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21e57d9-e303-4dd2-9a9f-a2bef26d5363 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:39.223958 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:39.260073 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594efca3-070d-4df3-8214-f5f7fd7d80f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:39.260704 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594efca3-070d-4df3-8214-f5f7fd7d80f2 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:39.260979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594efca3-070d-4df3-8214-f5f7fd7d80f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:39.261253 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594efca3-070d-4df3-8214-f5f7fd7d80f2 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:39.262149 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594efca3-070d-4df3-8214-f5f7fd7d80f2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:39.262455 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594efca3-070d-4df3-8214-f5f7fd7d80f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:39.264455 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594efca3-070d-4df3-8214-f5f7fd7d80f2 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:39.274023 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 429/853] 10.222.0.46 () {56 vars in 1126 bytes} [Wed Jul 27 04:09:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:39.276249 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594efca3-070d-4df3-8214-f5f7fd7d80f2 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:39.342252 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e0ec9-249a-455f-b616-1f2d85891550 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:39.377698 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cabc50-a50c-49cd-b373-d994cf10ddd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:39.378679 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cabc50-a50c-49cd-b373-d994cf10ddd3 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:39.379173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cabc50-a50c-49cd-b373-d994cf10ddd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:39.379655 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cabc50-a50c-49cd-b373-d994cf10ddd3 None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:39.380597 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cabc50-a50c-49cd-b373-d994cf10ddd3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:39.381126 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cabc50-a50c-49cd-b373-d994cf10ddd3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:39.382416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594efca3-070d-4df3-8214-f5f7fd7d80f2 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:39.383023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cabc50-a50c-49cd-b373-d994cf10ddd3 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:39.396306 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 425/854] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:09:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:09:39.402605 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cabc50-a50c-49cd-b373-d994cf10ddd3 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:39.429479 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f7467a-d29a-48fc-8e7c-f243092348e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:39.430140 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7467a-d29a-48fc-8e7c-f243092348e8 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:39.430421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7467a-d29a-48fc-8e7c-f243092348e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:39.430856 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7467a-d29a-48fc-8e7c-f243092348e8 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:39.431554 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f7467a-d29a-48fc-8e7c-f243092348e8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:39.431857 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f7467a-d29a-48fc-8e7c-f243092348e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:39.433767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f7467a-d29a-48fc-8e7c-f243092348e8 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:39.438775 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:39.446182 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 430/855] 10.222.0.46 () {56 vars in 1126 bytes} [Wed Jul 27 04:09:39 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:39.545116 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cabc50-a50c-49cd-b373-d994cf10ddd3 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:39.585521 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8af8d8-e4e6-49de-baf1-68171d5ee68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:39.586204 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8af8d8-e4e6-49de-baf1-68171d5ee68c None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:39.586483 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8af8d8-e4e6-49de-baf1-68171d5ee68c None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:39.586754 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8af8d8-e4e6-49de-baf1-68171d5ee68c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:39.587569 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8af8d8-e4e6-49de-baf1-68171d5ee68c None admin] RBAC: Adding query filter params (role.id=cf771a8e94ee46149eb081347fea6d67, scope.project.id=e646d320aa5a4f78bfccaf941cf46478, user.id=57b5fa7517034a39930ce7c9d12e1875) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:39.587875 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8af8d8-e4e6-49de-baf1-68171d5ee68c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:39.589335 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8af8d8-e4e6-49de-baf1-68171d5ee68c None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:39.596748 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 426/856] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:09:39 2022] GET /identity/v3/role_assignments?user.id=57b5fa7517034a39930ce7c9d12e1875&scope.project.id=e646d320aa5a4f78bfccaf941cf46478&role.id=cf771a8e94ee46149eb081347fea6d67 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:09:40.003891 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3e590-70c3-4776-9335-96b140815328 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:40.004410 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3e590-70c3-4776-9335-96b140815328 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:40.004515 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3e590-70c3-4776-9335-96b140815328 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:40.006043 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 431/857] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:40.011342 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290d12b-1dd6-42a0-ab10-b3d5cfa221cd None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:40.011813 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290d12b-1dd6-42a0-ab10-b3d5cfa221cd None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:40.012049 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290d12b-1dd6-42a0-ab10-b3d5cfa221cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:40.031028 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-d290d12b-1dd6-42a0-ab10-b3d5cfa221cd None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:40.032281 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d290d12b-1dd6-42a0-ab10-b3d5cfa221cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:40.103869 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 427/858] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:09:40.127137 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3e590-70c3-4776-9335-96b140815328 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:40.166313 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:40.233948 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3e590-70c3-4776-9335-96b140815328 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:40.273201 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cfdc7e-8987-43af-a8ac-8e029d37189a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:40.274236 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfdc7e-8987-43af-a8ac-8e029d37189a None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:40.274750 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfdc7e-8987-43af-a8ac-8e029d37189a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:40.275239 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfdc7e-8987-43af-a8ac-8e029d37189a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:40.286606 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cfdc7e-8987-43af-a8ac-8e029d37189a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:09:40.291777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-14cfdc7e-8987-43af-a8ac-8e029d37189a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:40.291777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:40.291777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:40.291777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:40.291777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:40.291777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:40.291777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:40.291777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:40.291777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:40.291777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:40.291777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:40.291777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:40.291777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:09:40.291777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:09:40.291777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:09:40.291777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:09:40.292735 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:09:40.292735 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:09:40.292735 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:09:40.292735 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:09:40.292735 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:40.292735 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:40.292735 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:40.292735 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:40.292735 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:40.292735 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:40.292735 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:40.292735 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:40.292735 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:40.292735 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:40.292735 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:40.292735 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:40.293580 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:40.293580 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:40.293580 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:40.293580 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:40.293580 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:09:40.293580 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:09:40.293580 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:09:40.293580 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:09:40.293580 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:09:40.293580 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:09:40.293580 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:09:40.293580 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:40.296242 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 432/859] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 27 04:09:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:09:40.302412 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d290d12b-1dd6-42a0-ab10-b3d5cfa221cd None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:40.321555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:40.412845 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d290d12b-1dd6-42a0-ab10-b3d5cfa221cd None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:40.448279 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cfe4fc-b0ba-4fe5-86d2-e78c5547eee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:40.448909 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cfe4fc-b0ba-4fe5-86d2-e78c5547eee0 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:40.449188 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cfe4fc-b0ba-4fe5-86d2-e78c5547eee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:40.449460 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cfe4fc-b0ba-4fe5-86d2-e78c5547eee0 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:40.450209 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cfe4fc-b0ba-4fe5-86d2-e78c5547eee0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:40.450511 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cfe4fc-b0ba-4fe5-86d2-e78c5547eee0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:40.451879 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cfe4fc-b0ba-4fe5-86d2-e78c5547eee0 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:40.460662 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 428/860] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 27 04:09:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:40.466031 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cfdc7e-8987-43af-a8ac-8e029d37189a None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:40.520432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d488b16-f330-4c9d-a0de-47a633e15f97 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:40.520797 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d488b16-f330-4c9d-a0de-47a633e15f97 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:40.520985 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d488b16-f330-4c9d-a0de-47a633e15f97 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:40.522660 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 429/861] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 04:09:40.527524 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c2052c-f883-46a2-8022-f82d17f18442 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:40.527755 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c2052c-f883-46a2-8022-f82d17f18442 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:40.528034 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c2052c-f883-46a2-8022-f82d17f18442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:40.548455 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-45c2052c-f883-46a2-8022-f82d17f18442 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:40.550806 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-45c2052c-f883-46a2-8022-f82d17f18442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:40.576859 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:40.584538 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cfdc7e-8987-43af-a8ac-8e029d37189a None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:40.626214 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18253622-c353-4e35-9346-228524893fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:40.626978 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18253622-c353-4e35-9346-228524893fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:40.627266 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18253622-c353-4e35-9346-228524893fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:40.627542 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18253622-c353-4e35-9346-228524893fd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:40.639487 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 430/862] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 04:09:40.642087 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18253622-c353-4e35-9346-228524893fd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:40.643872 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18253622-c353-4e35-9346-228524893fd7 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:40.660379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-18253622-c353-4e35-9346-228524893fd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:40.660379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:40.660379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:40.660379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:40.660379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:40.660379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:40.660379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:40.660379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:40.660379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:40.660379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:40.660379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:40.660379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:40.660379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:40.660379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:40.660379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:40.661372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:40.661372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:40.661372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:40.661372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:40.661372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:40.661372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:40.661372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:40.661372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:40.661372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:40.661372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:40.661372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:40.661372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:40.661372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:40.661372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:40.661372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:40.661372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:40.661372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:40.662364 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:40.662364 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:40.662364 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:40.662364 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:40.665235 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c2052c-f883-46a2-8022-f82d17f18442 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:40.666777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 433/863] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:40.673372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18253622-c353-4e35-9346-228524893fd7 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:40.682414 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:40.787017 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c2052c-f883-46a2-8022-f82d17f18442 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:40.787189 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18253622-c353-4e35-9346-228524893fd7 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:40.830994 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9984d73-6ccb-46dc-bcd4-48ba39fe4adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:40.831641 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9984d73-6ccb-46dc-bcd4-48ba39fe4adc None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:40.831927 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9984d73-6ccb-46dc-bcd4-48ba39fe4adc None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:40.832204 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9984d73-6ccb-46dc-bcd4-48ba39fe4adc None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:40.832917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9984d73-6ccb-46dc-bcd4-48ba39fe4adc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:40.833226 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9984d73-6ccb-46dc-bcd4-48ba39fe4adc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:40.834942 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9984d73-6ccb-46dc-bcd4-48ba39fe4adc None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:40.838006 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54412a45-ce00-42bb-a311-1e3da600b557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:40.838006 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54412a45-ce00-42bb-a311-1e3da600b557 None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:40.838006 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54412a45-ce00-42bb-a311-1e3da600b557 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:40.838006 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54412a45-ce00-42bb-a311-1e3da600b557 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:40.838782 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54412a45-ce00-42bb-a311-1e3da600b557 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:40.840345 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54412a45-ce00-42bb-a311-1e3da600b557 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:40.847984 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 434/864] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:40.853262 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9984d73-6ccb-46dc-bcd4-48ba39fe4adc None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:40.866062 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 431/865] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Jul 27 04:09:40 2022] POST /identity/v3/services => generated 251 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:09:40.873147 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:40.978937 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9984d73-6ccb-46dc-bcd4-48ba39fe4adc None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:41.029009 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78286d23-0bd5-467a-b487-cc02077dc35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:41.029784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78286d23-0bd5-467a-b487-cc02077dc35f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:41.029956 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78286d23-0bd5-467a-b487-cc02077dc35f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:41.030259 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78286d23-0bd5-467a-b487-cc02077dc35f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:41.038711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78286d23-0bd5-467a-b487-cc02077dc35f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:41.040595 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78286d23-0bd5-467a-b487-cc02077dc35f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:41.053372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-78286d23-0bd5-467a-b487-cc02077dc35f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 04:09:41.053372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:41.053372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:41.053372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:41.053372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:41.053372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:41.053372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:41.053372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:41.053372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:41.053372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:41.053372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:41.053372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:41.053372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:09:41.053372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:09:41.053372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:09:41.053372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:09:41.054411 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:41.054411 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:41.054411 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:09:41.054411 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:41.054411 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:09:41.054411 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:41.054411 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:41.054411 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:41.054411 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:41.054411 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:41.054411 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:41.054411 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:41.054411 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:41.054411 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:41.054411 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:41.054411 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:41.054411 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:41.054411 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:41.055356 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:41.055356 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:41.055356 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:09:41.055356 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:09:41.055356 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:09:41.055356 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:09:41.055356 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:09:41.055356 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:09:41.055356 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 04:09:41.055356 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:41.055356 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 435/866] 10.222.0.46 () {56 vars in 1117 bytes} [Wed Jul 27 04:09:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:41.062070 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54412a45-ce00-42bb-a311-1e3da600b557 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:41.140331 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:41.175069 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54412a45-ce00-42bb-a311-1e3da600b557 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:41.212228 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4fcf8-f4ce-4ddf-8636-bd28d6c8faea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:41.213248 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4fcf8-f4ce-4ddf-8636-bd28d6c8faea None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:41.213742 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4fcf8-f4ce-4ddf-8636-bd28d6c8faea None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:41.214319 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4fcf8-f4ce-4ddf-8636-bd28d6c8faea None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:41.215350 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4fcf8-f4ce-4ddf-8636-bd28d6c8faea None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:41.215880 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4fcf8-f4ce-4ddf-8636-bd28d6c8faea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:41.217531 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4fcf8-f4ce-4ddf-8636-bd28d6c8faea None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:41.237066 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 432/867] 10.222.0.46 () {56 vars in 1149 bytes} [Wed Jul 27 04:09:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:41.242534 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78286d23-0bd5-467a-b487-cc02077dc35f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:41.309089 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:41.352660 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78286d23-0bd5-467a-b487-cc02077dc35f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:41.388772 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e9af8-237b-42bd-9f68-93584d1d28be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:41.389410 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e9af8-237b-42bd-9f68-93584d1d28be None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:41.389697 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e9af8-237b-42bd-9f68-93584d1d28be None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:41.389991 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e9af8-237b-42bd-9f68-93584d1d28be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:41.465182 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e9af8-237b-42bd-9f68-93584d1d28be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:41.466455 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e9af8-237b-42bd-9f68-93584d1d28be None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:41.530233 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:41.574434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 436/868] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:09:41 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 04:09:41.580660 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4fcf8-f4ce-4ddf-8636-bd28d6c8faea None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:41.582965 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d28b8-35b2-4bf5-bdb5-2940f7aa015e None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:41.583253 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d28b8-35b2-4bf5-bdb5-2940f7aa015e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:41.583533 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d28b8-35b2-4bf5-bdb5-2940f7aa015e None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:41.585112 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 437/869] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 04:09:41.590001 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850813bf-bda5-4bb7-8b53-c17ae6976087 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:41.590289 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850813bf-bda5-4bb7-8b53-c17ae6976087 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:41.590560 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850813bf-bda5-4bb7-8b53-c17ae6976087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:41.615060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-850813bf-bda5-4bb7-8b53-c17ae6976087 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:41.617105 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-850813bf-bda5-4bb7-8b53-c17ae6976087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:41.694565 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 438/870] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:41 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 04:09:41.695659 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4fcf8-f4ce-4ddf-8636-bd28d6c8faea None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:41.718438 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850813bf-bda5-4bb7-8b53-c17ae6976087 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:41.723982 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:41.732678 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811afc40-3dea-4e47-80f4-700b69b9607b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:41.733334 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811afc40-3dea-4e47-80f4-700b69b9607b None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:41.733617 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811afc40-3dea-4e47-80f4-700b69b9607b None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:41.733894 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811afc40-3dea-4e47-80f4-700b69b9607b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:41.745125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811afc40-3dea-4e47-80f4-700b69b9607b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:41.746670 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811afc40-3dea-4e47-80f4-700b69b9607b None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:41.762902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-811afc40-3dea-4e47-80f4-700b69b9607b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:41.762902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:41.762902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:41.762902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:41.762902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:41.762902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:41.762902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:41.762902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:41.762902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:41.762902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:41.762902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:41.762902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:41.762902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:41.762902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:41.762902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:41.763994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:41.763994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:41.763994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:41.763994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:41.763994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:41.763994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:41.763994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:41.763994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:41.763994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:41.763994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:41.763994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:41.763994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:41.763994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:41.763994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:41.763994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:41.763994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:41.763994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:41.764891 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:41.764891 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:41.764891 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:41.764891 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:41.764891 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 433/871] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:41.765540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fab0b-20ae-4d90-ad34-ac1a5710b083 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:41.765540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fab0b-20ae-4d90-ad34-ac1a5710b083 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:41.765861 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fab0b-20ae-4d90-ad34-ac1a5710b083 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:41.767317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 434/872] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:41.770869 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6fab0b-20ae-4d90-ad34-ac1a5710b083 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:41.840570 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:41.849097 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850813bf-bda5-4bb7-8b53-c17ae6976087 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:41.881749 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6fab0b-20ae-4d90-ad34-ac1a5710b083 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:41.885303 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e837e-826e-47df-94f2-0de7811978ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:41.886086 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e837e-826e-47df-94f2-0de7811978ad None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:41.886299 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e837e-826e-47df-94f2-0de7811978ad None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:41.886576 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e837e-826e-47df-94f2-0de7811978ad None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:41.887285 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8e837e-826e-47df-94f2-0de7811978ad None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:41.888655 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8e837e-826e-47df-94f2-0de7811978ad None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:41.905242 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 439/873] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Jul 27 04:09:41 2022] POST /identity/v3/services => generated 269 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:09:41.907417 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9e5c2-c3a9-4efb-b1bc-97d274d89c76 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:41.907811 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9e5c2-c3a9-4efb-b1bc-97d274d89c76 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:41.908097 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9e5c2-c3a9-4efb-b1bc-97d274d89c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:41.928690 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-82f9e5c2-c3a9-4efb-b1bc-97d274d89c76 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:41.929941 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-82f9e5c2-c3a9-4efb-b1bc-97d274d89c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:41.937034 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a24e7aa-aa44-4f7b-bccb-957f5007d999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:41.939173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a24e7aa-aa44-4f7b-bccb-957f5007d999 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:41.939862 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a24e7aa-aa44-4f7b-bccb-957f5007d999 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:41.940350 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a24e7aa-aa44-4f7b-bccb-957f5007d999 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:41.941311 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a24e7aa-aa44-4f7b-bccb-957f5007d999 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:41.941850 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a24e7aa-aa44-4f7b-bccb-957f5007d999 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:41.943666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a24e7aa-aa44-4f7b-bccb-957f5007d999 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:41.957924 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 435/874] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:41.964087 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a24e7aa-aa44-4f7b-bccb-957f5007d999 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:42.012648 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:42.042388 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 440/875] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:41 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:09:42.066417 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f9e5c2-c3a9-4efb-b1bc-97d274d89c76 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:42.102975 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a24e7aa-aa44-4f7b-bccb-957f5007d999 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:42.150971 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7fdf7-306a-43a3-969e-a7c50b1d628a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:42.151705 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7fdf7-306a-43a3-969e-a7c50b1d628a None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:42.151905 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7fdf7-306a-43a3-969e-a7c50b1d628a None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:42.152182 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7fdf7-306a-43a3-969e-a7c50b1d628a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:42.160347 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7fdf7-306a-43a3-969e-a7c50b1d628a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:42.161805 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7fdf7-306a-43a3-969e-a7c50b1d628a None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:42.171895 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:42.174301 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-22d7fdf7-306a-43a3-969e-a7c50b1d628a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:42.174301 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:42.174301 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:42.174301 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:42.174301 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:42.174301 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:42.174301 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:42.174301 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:42.174301 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:42.174301 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:42.174301 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:42.174301 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:42.174301 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:09:42.174301 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:09:42.174301 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:09:42.175272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:09:42.175272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:42.175272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:42.175272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:42.175272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:42.175272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:42.175272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:42.175272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:42.175272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:42.175272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:42.175272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:42.175272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:42.175272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:42.175272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:42.175272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:42.175272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:42.175272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:42.176273 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:09:42.176273 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:09:42.176273 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:09:42.176273 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:09:42.176273 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:09:42.176273 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:09:42.176273 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:42.176273 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:42.176273 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 436/876] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:42.182036 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7fdf7-306a-43a3-969e-a7c50b1d628a None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:42.195537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f9e5c2-c3a9-4efb-b1bc-97d274d89c76 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:42.243486 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad7e7e2-81ef-45ac-886d-0e7498323d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:42.244323 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7e7e2-81ef-45ac-886d-0e7498323d14 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:42.244706 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7e7e2-81ef-45ac-886d-0e7498323d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:42.245088 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7e7e2-81ef-45ac-886d-0e7498323d14 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:42.245910 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad7e7e2-81ef-45ac-886d-0e7498323d14 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:42.247334 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad7e7e2-81ef-45ac-886d-0e7498323d14 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:42.257077 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-3ad7e7e2-81ef-45ac-886d-0e7498323d14 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 04:09:42.257077 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:42.257077 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 04:09:42.257077 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 04:09:42.257077 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 04:09:42.257077 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 04:09:42.257077 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 04:09:42.257077 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:42.257077 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 04:09:42.257077 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:42.257077 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 04:09:42.257077 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:42.257077 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:42.257077 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:42.257077 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:42.257077 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:42.257077 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:42.258322 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:42.258322 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:42.258322 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:42.258322 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:42.258322 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:42.258322 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:42.258322 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 04:09:42.258322 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 04:09:42.258322 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 04:09:42.258322 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 04:09:42.258322 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:42.258322 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:42.258322 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:42.258322 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:42.258322 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:42.258322 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:42.259372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:42.259372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:42.259372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:42.259372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:42.259372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:42.259372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:42.259372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:42.259372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:42.259372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:42.259372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:42.259372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 04:09:42.259372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:42.259372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 04:09:42.259372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:42.261354 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 441/877] 10.222.0.46 () {56 vars in 1126 bytes} [Wed Jul 27 04:09:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:09:42.268202 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad7e7e2-81ef-45ac-886d-0e7498323d14 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:42.303284 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:42.310245 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7fdf7-306a-43a3-969e-a7c50b1d628a None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:42.346614 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff99574d-60c1-4de2-865a-d37f0dbab996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:42.347487 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99574d-60c1-4de2-865a-d37f0dbab996 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:42.347918 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99574d-60c1-4de2-865a-d37f0dbab996 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:42.348342 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99574d-60c1-4de2-865a-d37f0dbab996 None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:42.349246 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff99574d-60c1-4de2-865a-d37f0dbab996 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:42.349725 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff99574d-60c1-4de2-865a-d37f0dbab996 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:42.351357 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff99574d-60c1-4de2-865a-d37f0dbab996 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:42.364852 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 437/878] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:09:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:42.374091 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff99574d-60c1-4de2-865a-d37f0dbab996 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:42.379238 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad7e7e2-81ef-45ac-886d-0e7498323d14 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:42.416034 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58770c4c-7cf9-4746-814c-69884fd6c043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:42.416855 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58770c4c-7cf9-4746-814c-69884fd6c043 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:42.417279 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58770c4c-7cf9-4746-814c-69884fd6c043 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:42.417681 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58770c4c-7cf9-4746-814c-69884fd6c043 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:42.418506 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58770c4c-7cf9-4746-814c-69884fd6c043 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:42.418915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58770c4c-7cf9-4746-814c-69884fd6c043 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:42.420371 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58770c4c-7cf9-4746-814c-69884fd6c043 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:42.430229 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 442/879] 10.222.0.46 () {56 vars in 1122 bytes} [Wed Jul 27 04:09:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:42.432348 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58770c4c-7cf9-4746-814c-69884fd6c043 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:42.450089 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:42.520163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff99574d-60c1-4de2-865a-d37f0dbab996 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:42.557844 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a5cf30-3a7a-46d4-8d96-66ef2a7a2147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:42.558800 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a5cf30-3a7a-46d4-8d96-66ef2a7a2147 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:42.559269 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a5cf30-3a7a-46d4-8d96-66ef2a7a2147 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:42.559753 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a5cf30-3a7a-46d4-8d96-66ef2a7a2147 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:42.560744 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a5cf30-3a7a-46d4-8d96-66ef2a7a2147 None admin] RBAC: Adding query filter params (role.id=6521132834a64b029695bbdd5eb251d2, scope.project.id=e646d320aa5a4f78bfccaf941cf46478, user.id=8ec07ee1efba4e8089b94f24be338f86) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:42.561277 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a5cf30-3a7a-46d4-8d96-66ef2a7a2147 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:42.563810 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a5cf30-3a7a-46d4-8d96-66ef2a7a2147 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:42.578045 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58770c4c-7cf9-4746-814c-69884fd6c043 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:42.578319 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 438/880] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:09:42 2022] GET /identity/v3/role_assignments?user.id=8ec07ee1efba4e8089b94f24be338f86&scope.project.id=e646d320aa5a4f78bfccaf941cf46478&role.id=6521132834a64b029695bbdd5eb251d2 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:42.610264 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dadcce-5eeb-4641-9388-a3c11e37a840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:42.610920 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dadcce-5eeb-4641-9388-a3c11e37a840 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:42.611195 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dadcce-5eeb-4641-9388-a3c11e37a840 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:42.611471 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dadcce-5eeb-4641-9388-a3c11e37a840 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:42.612179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dadcce-5eeb-4641-9388-a3c11e37a840 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:42.612487 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dadcce-5eeb-4641-9388-a3c11e37a840 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:42.613869 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dadcce-5eeb-4641-9388-a3c11e37a840 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:42.619634 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 443/881] 10.222.0.46 () {56 vars in 1122 bytes} [Wed Jul 27 04:09:42 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:09:42.999901 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963bf0b8-8172-419c-ac7a-4b2736db56f1 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:43.000092 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963bf0b8-8172-419c-ac7a-4b2736db56f1 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:43.001001 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963bf0b8-8172-419c-ac7a-4b2736db56f1 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:43.002624 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 439/882] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:43.007830 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4990a-5c16-4234-8c2c-cebed7fcaa2b None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:43.008119 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4990a-5c16-4234-8c2c-cebed7fcaa2b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:43.008398 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4990a-5c16-4234-8c2c-cebed7fcaa2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:43.034688 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-e7a4990a-5c16-4234-8c2c-cebed7fcaa2b None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:43.036328 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e7a4990a-5c16-4234-8c2c-cebed7fcaa2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:43.122186 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 444/883] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:43 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:09:43.145330 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963bf0b8-8172-419c-ac7a-4b2736db56f1 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:43.168394 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:43.259027 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963bf0b8-8172-419c-ac7a-4b2736db56f1 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:43.299594 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0386e-7a35-48e5-9084-50e267cb75d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:43.300249 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0386e-7a35-48e5-9084-50e267cb75d3 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:43.300533 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0386e-7a35-48e5-9084-50e267cb75d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:43.300809 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0386e-7a35-48e5-9084-50e267cb75d3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:43.301452 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf0386e-7a35-48e5-9084-50e267cb75d3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:43.302861 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf0386e-7a35-48e5-9084-50e267cb75d3 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:43.311900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-baf0386e-7a35-48e5-9084-50e267cb75d3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 04:09:43.311900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:43.311900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 04:09:43.311900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 04:09:43.311900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 04:09:43.311900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 04:09:43.311900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 04:09:43.311900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:43.311900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 04:09:43.311900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:43.311900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 04:09:43.311900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:43.311900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:43.311900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:43.311900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:43.311900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:43.311900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:43.312919 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:43.312919 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:43.312919 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:43.312919 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:43.312919 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:43.312919 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:43.312919 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 04:09:43.312919 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 04:09:43.312919 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 04:09:43.312919 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 04:09:43.312919 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:43.312919 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:43.312919 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:43.312919 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:43.312919 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:43.312919 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:43.314037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:43.314037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:43.314037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:43.314037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:43.314037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:43.314037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:43.314037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:43.314037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:43.314037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:43.314037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:43.314037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 04:09:43.314037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:43.314037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 04:09:43.314037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:43.314037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 440/884] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 27 04:09:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:43.321174 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4990a-5c16-4234-8c2c-cebed7fcaa2b None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:43.330041 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:43.429278 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4990a-5c16-4234-8c2c-cebed7fcaa2b None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:43.468251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c07a7-1f2a-4eb3-bac9-7383a5cc2edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:43.469424 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c07a7-1f2a-4eb3-bac9-7383a5cc2edc None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:43.469715 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c07a7-1f2a-4eb3-bac9-7383a5cc2edc None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:43.470019 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c07a7-1f2a-4eb3-bac9-7383a5cc2edc None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:43.470766 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620c07a7-1f2a-4eb3-bac9-7383a5cc2edc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:43.471074 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620c07a7-1f2a-4eb3-bac9-7383a5cc2edc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:43.472451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620c07a7-1f2a-4eb3-bac9-7383a5cc2edc None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:43.477960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 445/885] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 27 04:09:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:43.487040 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0386e-7a35-48e5-9084-50e267cb75d3 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:43.576466 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:43.592256 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0386e-7a35-48e5-9084-50e267cb75d3 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:43.628132 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ce5d5-2f60-4320-a280-cb5fb8e9ca46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:43.628758 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ce5d5-2f60-4320-a280-cb5fb8e9ca46 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:43.629045 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ce5d5-2f60-4320-a280-cb5fb8e9ca46 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:43.629323 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ce5d5-2f60-4320-a280-cb5fb8e9ca46 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:43.630035 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ce5d5-2f60-4320-a280-cb5fb8e9ca46 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:43.630348 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ce5d5-2f60-4320-a280-cb5fb8e9ca46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:43.631725 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ce5d5-2f60-4320-a280-cb5fb8e9ca46 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:43.637142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 441/886] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 27 04:09:43 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:43.642884 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c07a7-1f2a-4eb3-bac9-7383a5cc2edc None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:43.718007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:43.746480 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c07a7-1f2a-4eb3-bac9-7383a5cc2edc None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:43.781764 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2cf9ae-832f-4b46-862d-8d9904e2afed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:43.782804 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2cf9ae-832f-4b46-862d-8d9904e2afed None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:43.783294 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2cf9ae-832f-4b46-862d-8d9904e2afed None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:43.783735 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2cf9ae-832f-4b46-862d-8d9904e2afed None admin] PATH_INFO: `/v3/endpoints` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:43.785083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2cf9ae-832f-4b46-862d-8d9904e2afed None admin] RBAC: Adding query filter params (interface=public, service_id=83ef63bc6cbc4f879ded6e72854cd1f0, region_id=RegionOne) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:43.785611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2cf9ae-832f-4b46-862d-8d9904e2afed None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:43.788197 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2cf9ae-832f-4b46-862d-8d9904e2afed None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:43.798208 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 446/887] 10.222.0.46 () {56 vars in 1251 bytes} [Wed Jul 27 04:09:43 2022] GET /identity/v3/endpoints?service_id=83ef63bc6cbc4f879ded6e72854cd1f0&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:43.803629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ce5d5-2f60-4320-a280-cb5fb8e9ca46 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:43.922441 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ce5d5-2f60-4320-a280-cb5fb8e9ca46 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:43.958786 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb71a6a2-2884-424c-b0f3-0f23a8c5b5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:43.959413 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71a6a2-2884-424c-b0f3-0f23a8c5b5b2 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:43.959693 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71a6a2-2884-424c-b0f3-0f23a8c5b5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:43.960029 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71a6a2-2884-424c-b0f3-0f23a8c5b5b2 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:43.960697 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb71a6a2-2884-424c-b0f3-0f23a8c5b5b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:43.962159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb71a6a2-2884-424c-b0f3-0f23a8c5b5b2 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:43.968421 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 442/888] 10.222.0.46 () {56 vars in 1087 bytes} [Wed Jul 27 04:09:43 2022] GET /identity/v3/services => generated 1268 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:09:44.126309 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf20a7-4530-41f1-a0e5-937d707ea10c None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:44.126812 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf20a7-4530-41f1-a0e5-937d707ea10c None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:44.126916 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf20a7-4530-41f1-a0e5-937d707ea10c None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:44.128333 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 447/889] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:44.133720 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f255ed-436a-446e-aa9a-a6a24a397b9c None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:44.134034 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f255ed-436a-446e-aa9a-a6a24a397b9c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:44.134320 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f255ed-436a-446e-aa9a-a6a24a397b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:44.157600 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-07f255ed-436a-446e-aa9a-a6a24a397b9c None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:44.158843 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-07f255ed-436a-446e-aa9a-a6a24a397b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:44.167097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:44.234125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 443/890] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:44 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:09:44.256923 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf20a7-4530-41f1-a0e5-937d707ea10c None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:44.366542 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf20a7-4530-41f1-a0e5-937d707ea10c None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:44.402658 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790fe530-5a86-4e02-9acc-69b208ede201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:44.403317 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790fe530-5a86-4e02-9acc-69b208ede201 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:44.403643 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790fe530-5a86-4e02-9acc-69b208ede201 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:44.403924 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790fe530-5a86-4e02-9acc-69b208ede201 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:44.404563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790fe530-5a86-4e02-9acc-69b208ede201 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:44.405934 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790fe530-5a86-4e02-9acc-69b208ede201 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:44.407929 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:44.416267 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-790fe530-5a86-4e02-9acc-69b208ede201 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 04:09:44.416267 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:44.416267 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 04:09:44.416267 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 04:09:44.416267 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 04:09:44.416267 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 04:09:44.416267 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 04:09:44.416267 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:44.416267 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 04:09:44.416267 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:44.416267 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 04:09:44.416267 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:44.416267 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:44.416267 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:44.416267 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:44.416267 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:44.417222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:44.417222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:44.417222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:44.417222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:44.417222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:44.417222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:44.417222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:44.417222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 04:09:44.417222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 04:09:44.417222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 04:09:44.417222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 04:09:44.417222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:44.417222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:44.417222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:44.417222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:44.417222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:44.418087 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:44.418087 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:44.418087 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:44.418087 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:44.418087 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:44.418087 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:44.418087 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:44.418087 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:44.418087 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:44.418087 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:44.418087 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:44.418087 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 04:09:44.418087 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:44.418087 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 04:09:44.418087 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:44.418087 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 448/891] 10.222.0.46 () {56 vars in 1132 bytes} [Wed Jul 27 04:09:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:44.425796 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f255ed-436a-446e-aa9a-a6a24a397b9c None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:44.537451 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f255ed-436a-446e-aa9a-a6a24a397b9c None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:44.572171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc11d03-e9f3-46ca-a5d4-c5ec67b426ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:44.572797 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc11d03-e9f3-46ca-a5d4-c5ec67b426ad None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:44.573076 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc11d03-e9f3-46ca-a5d4-c5ec67b426ad None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:44.573345 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc11d03-e9f3-46ca-a5d4-c5ec67b426ad None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:44.574053 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc11d03-e9f3-46ca-a5d4-c5ec67b426ad None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:44.574358 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc11d03-e9f3-46ca-a5d4-c5ec67b426ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:44.575780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc11d03-e9f3-46ca-a5d4-c5ec67b426ad None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:44.581370 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 444/892] 10.222.0.46 () {56 vars in 1126 bytes} [Wed Jul 27 04:09:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:44.586994 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790fe530-5a86-4e02-9acc-69b208ede201 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:44.692690 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790fe530-5a86-4e02-9acc-69b208ede201 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:44.733118 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09976524-a3bb-42f1-b2ca-c733d708f024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:44.733748 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09976524-a3bb-42f1-b2ca-c733d708f024 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:44.734059 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09976524-a3bb-42f1-b2ca-c733d708f024 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:44.734342 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09976524-a3bb-42f1-b2ca-c733d708f024 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:44.735582 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09976524-a3bb-42f1-b2ca-c733d708f024 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:44.735888 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09976524-a3bb-42f1-b2ca-c733d708f024 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:44.737261 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09976524-a3bb-42f1-b2ca-c733d708f024 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:44.742874 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 449/893] 10.222.0.46 () {56 vars in 1126 bytes} [Wed Jul 27 04:09:44 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:44.748657 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc11d03-e9f3-46ca-a5d4-c5ec67b426ad None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:44.758711 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:44.813740 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a18fb-36be-4202-af27-f6f1c15a2a7b None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:44.814212 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a18fb-36be-4202-af27-f6f1c15a2a7b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:44.814334 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a18fb-36be-4202-af27-f6f1c15a2a7b None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:44.815899 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 450/894] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 04:09:44.820696 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf97dd-8211-42f0-bc84-93dca254450c None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:44.820987 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf97dd-8211-42f0-bc84-93dca254450c None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:44.821273 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf97dd-8211-42f0-bc84-93dca254450c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:44.840558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-7bcf97dd-8211-42f0-bc84-93dca254450c None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:44.841752 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7bcf97dd-8211-42f0-bc84-93dca254450c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:44.872163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc11d03-e9f3-46ca-a5d4-c5ec67b426ad None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:44.905468 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:44.911354 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a13785-3c80-44d1-b9ac-f340c6f8a351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:44.911980 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a13785-3c80-44d1-b9ac-f340c6f8a351 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:44.912265 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a13785-3c80-44d1-b9ac-f340c6f8a351 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:44.912538 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a13785-3c80-44d1-b9ac-f340c6f8a351 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:44.913404 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a13785-3c80-44d1-b9ac-f340c6f8a351 None admin] RBAC: Adding query filter params (interface=public, service_id=76e6cd8690034b9b8c9c5b821a50ee33, region_id=RegionOne) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:44.913617 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a13785-3c80-44d1-b9ac-f340c6f8a351 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:44.914995 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a13785-3c80-44d1-b9ac-f340c6f8a351 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:44.918250 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 451/895] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:44 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 04:09:44.919365 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171a059-b815-4a74-a6c2-0823da054919 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:44.919654 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171a059-b815-4a74-a6c2-0823da054919 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:44.919930 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171a059-b815-4a74-a6c2-0823da054919 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:44.925790 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 452/896] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:44.926869 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5a99d-bf3e-4159-991c-f0c52b6f6b9c None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:44.927150 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5a99d-bf3e-4159-991c-f0c52b6f6b9c None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:44.927420 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5a99d-bf3e-4159-991c-f0c52b6f6b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:44.935882 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 445/897] 10.222.0.46 () {56 vars in 1251 bytes} [Wed Jul 27 04:09:44 2022] GET /identity/v3/endpoints?service_id=76e6cd8690034b9b8c9c5b821a50ee33&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:44.937314 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a13785-3c80-44d1-b9ac-f340c6f8a351 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:44.955925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-3df5a99d-bf3e-4159-991c-f0c52b6f6b9c None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:44.957132 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-3df5a99d-bf3e-4159-991c-f0c52b6f6b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:45.013024 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:45.028432 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 453/898] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:44 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 04:09:45.034254 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5a99d-bf3e-4159-991c-f0c52b6f6b9c None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:45.054939 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a13785-3c80-44d1-b9ac-f340c6f8a351 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:45.090645 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca01620-f9e5-4e20-ac67-3bf007d4b768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:45.091285 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca01620-f9e5-4e20-ac67-3bf007d4b768 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:45.091582 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca01620-f9e5-4e20-ac67-3bf007d4b768 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:45.091841 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca01620-f9e5-4e20-ac67-3bf007d4b768 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:45.092505 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca01620-f9e5-4e20-ac67-3bf007d4b768 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:45.094422 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca01620-f9e5-4e20-ac67-3bf007d4b768 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:45.110002 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 446/899] 10.222.0.46 () {56 vars in 1087 bytes} [Wed Jul 27 04:09:44 2022] GET /identity/v3/services => generated 1268 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 04:09:45.110002 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca01620-f9e5-4e20-ac67-3bf007d4b768 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:45.165539 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5a99d-bf3e-4159-991c-f0c52b6f6b9c None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:45.198911 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:45.214145 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42e360-8d2b-42c4-9bb7-a577fd26c5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:45.214860 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42e360-8d2b-42c4-9bb7-a577fd26c5e3 None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:45.215144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42e360-8d2b-42c4-9bb7-a577fd26c5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:45.215422 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42e360-8d2b-42c4-9bb7-a577fd26c5e3 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:45.216301 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e42e360-8d2b-42c4-9bb7-a577fd26c5e3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:45.217724 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e42e360-8d2b-42c4-9bb7-a577fd26c5e3 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:45.241441 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 454/900] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Jul 27 04:09:45 2022] POST /identity/v3/services => generated 241 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:09:45.248278 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca01620-f9e5-4e20-ac67-3bf007d4b768 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:45.303616 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:45.312720 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fc55e1-97a4-460f-b960-6034f0b0a601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:45.313730 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc55e1-97a4-460f-b960-6034f0b0a601 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:45.315032 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc55e1-97a4-460f-b960-6034f0b0a601 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:45.315032 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc55e1-97a4-460f-b960-6034f0b0a601 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:45.332044 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fc55e1-97a4-460f-b960-6034f0b0a601 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:09:45.337712 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-86fc55e1-97a4-460f-b960-6034f0b0a601 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 04:09:45.337712 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:45.337712 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:45.337712 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:45.337712 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:45.337712 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:45.337712 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:45.337712 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:45.337712 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:45.337712 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:45.337712 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:45.337712 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:45.337712 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:09:45.337712 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:09:45.337712 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:09:45.337712 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:09:45.338851 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:09:45.338851 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:09:45.338851 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:09:45.338851 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:09:45.338851 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:45.338851 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:45.338851 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:45.338851 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:45.338851 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:45.338851 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:45.338851 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:45.338851 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:45.338851 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:45.338851 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:45.338851 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:45.338851 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:45.339743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:45.339743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:45.339743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:45.339743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:45.339743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:09:45.339743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:09:45.339743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:09:45.339743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:09:45.339743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:09:45.339743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:09:45.339743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 04:09:45.339743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:45.339743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 447/901] 10.222.0.46 () {56 vars in 1120 bytes} [Wed Jul 27 04:09:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:45.345435 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42e360-8d2b-42c4-9bb7-a577fd26c5e3 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:45.461757 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42e360-8d2b-42c4-9bb7-a577fd26c5e3 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:45.495004 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b274c2b4-d65d-4d76-9e6a-085129ad7a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:45.495697 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274c2b4-d65d-4d76-9e6a-085129ad7a6c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:45.495989 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274c2b4-d65d-4d76-9e6a-085129ad7a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:45.496310 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274c2b4-d65d-4d76-9e6a-085129ad7a6c None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:45.497040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b274c2b4-d65d-4d76-9e6a-085129ad7a6c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:45.497344 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b274c2b4-d65d-4d76-9e6a-085129ad7a6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:45.498831 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b274c2b4-d65d-4d76-9e6a-085129ad7a6c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:45.508218 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 455/902] 10.222.0.46 () {56 vars in 1115 bytes} [Wed Jul 27 04:09:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:45.513981 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fc55e1-97a4-460f-b960-6034f0b0a601 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:45.624811 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fc55e1-97a4-460f-b960-6034f0b0a601 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:45.661799 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941567e1-db4c-4f87-810f-5a1246e5c865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:45.662780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941567e1-db4c-4f87-810f-5a1246e5c865 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:45.663239 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941567e1-db4c-4f87-810f-5a1246e5c865 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:45.663688 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941567e1-db4c-4f87-810f-5a1246e5c865 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:45.676254 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941567e1-db4c-4f87-810f-5a1246e5c865 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:45.677949 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941567e1-db4c-4f87-810f-5a1246e5c865 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:45.693608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-941567e1-db4c-4f87-810f-5a1246e5c865 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:45.693608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:45.693608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:45.693608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:45.693608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:45.693608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:45.693608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:45.693608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:45.693608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:45.693608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:45.693608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:45.693608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:45.693608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:45.693608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:45.693608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:45.694558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:45.694558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:45.694558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:45.694558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:45.694558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:45.694558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:45.694558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:45.694558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:45.694558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:45.694558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:45.694558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:45.694558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:45.694558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:45.694558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:45.694558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:45.694558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:45.694558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:45.695428 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:45.695428 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:45.695428 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:45.695428 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:45.697304 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 448/903] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:09:45.703023 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b274c2b4-d65d-4d76-9e6a-085129ad7a6c None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:45.754483 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:45.817459 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b274c2b4-d65d-4d76-9e6a-085129ad7a6c None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:45.853466 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2465989b-df2d-4b22-9d9c-908ac120a826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:45.854408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465989b-df2d-4b22-9d9c-908ac120a826 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:45.854835 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465989b-df2d-4b22-9d9c-908ac120a826 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:45.855257 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465989b-df2d-4b22-9d9c-908ac120a826 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:45.856165 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2465989b-df2d-4b22-9d9c-908ac120a826 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:45.856613 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2465989b-df2d-4b22-9d9c-908ac120a826 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:45.858222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2465989b-df2d-4b22-9d9c-908ac120a826 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:45.871488 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 456/904] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:45.877794 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941567e1-db4c-4f87-810f-5a1246e5c865 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:45.926362 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:45.985783 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941567e1-db4c-4f87-810f-5a1246e5c865 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:46.019977 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db0c64-b41b-4f81-b350-fd27c188b3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:46.020626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db0c64-b41b-4f81-b350-fd27c188b3a1 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:46.020905 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db0c64-b41b-4f81-b350-fd27c188b3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:46.021177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db0c64-b41b-4f81-b350-fd27c188b3a1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:46.028823 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db0c64-b41b-4f81-b350-fd27c188b3a1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:46.030278 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db0c64-b41b-4f81-b350-fd27c188b3a1 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:46.041567 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-90db0c64-b41b-4f81-b350-fd27c188b3a1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 04:09:46.041567 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:46.041567 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:46.041567 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:46.041567 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:46.041567 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:46.041567 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:46.041567 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:46.041567 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:46.041567 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:46.041567 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:46.041567 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:46.041567 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:09:46.041567 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:09:46.041567 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:09:46.041567 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:09:46.042574 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:46.042574 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:46.042574 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:09:46.042574 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:46.042574 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:09:46.042574 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:46.042574 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:46.042574 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:46.042574 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:46.042574 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:46.042574 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:46.042574 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:46.042574 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:46.042574 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:46.042574 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:46.042574 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:46.042574 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:46.042574 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:46.043695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:46.043695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:46.043695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:09:46.043695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:09:46.043695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:09:46.043695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:09:46.043695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:09:46.043695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:09:46.043695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 04:09:46.043695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:46.043695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 449/905] 10.222.0.46 () {56 vars in 1117 bytes} [Wed Jul 27 04:09:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:46.051161 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2465989b-df2d-4b22-9d9c-908ac120a826 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:46.107038 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea251c02-64e6-491c-a50a-19bad9e0dcb3 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:46.107394 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea251c02-64e6-491c-a50a-19bad9e0dcb3 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:46.107585 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea251c02-64e6-491c-a50a-19bad9e0dcb3 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:46.109047 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 450/906] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 04:09:46.114894 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bd18e-a4da-4051-9324-754c5ab5af45 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:46.115175 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bd18e-a4da-4051-9324-754c5ab5af45 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:46.115445 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bd18e-a4da-4051-9324-754c5ab5af45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:46.138530 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-fe4bd18e-a4da-4051-9324-754c5ab5af45 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:46.140144 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-fe4bd18e-a4da-4051-9324-754c5ab5af45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:46.168489 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2465989b-df2d-4b22-9d9c-908ac120a826 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:46.173128 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:46.209056 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6bf7c9-6d82-4d6d-b99b-b79899a49e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:46.210089 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6bf7c9-6d82-4d6d-b99b-b79899a49e81 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:46.210576 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6bf7c9-6d82-4d6d-b99b-b79899a49e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:46.211053 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6bf7c9-6d82-4d6d-b99b-b79899a49e81 None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:46.212481 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6bf7c9-6d82-4d6d-b99b-b79899a49e81 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:46.213220 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6bf7c9-6d82-4d6d-b99b-b79899a49e81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:46.215182 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6bf7c9-6d82-4d6d-b99b-b79899a49e81 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:46.227447 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 451/907] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 04:09:46.238463 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 457/908] 10.222.0.46 () {56 vars in 1149 bytes} [Wed Jul 27 04:09:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:46.247097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4bd18e-a4da-4051-9324-754c5ab5af45 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:46.251346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6bf7c9-6d82-4d6d-b99b-b79899a49e81 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:46.285427 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:46.361653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4bd18e-a4da-4051-9324-754c5ab5af45 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:46.372534 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6bf7c9-6d82-4d6d-b99b-b79899a49e81 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:46.396993 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b66658d-6b3c-400b-a4ff-265cec819048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:46.397460 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b66658d-6b3c-400b-a4ff-265cec819048 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:46.397749 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b66658d-6b3c-400b-a4ff-265cec819048 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:46.398003 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b66658d-6b3c-400b-a4ff-265cec819048 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:46.414252 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed28ca97-335c-4cbb-919a-2fe8301addd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:46.415343 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28ca97-335c-4cbb-919a-2fe8301addd9 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:46.415860 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28ca97-335c-4cbb-919a-2fe8301addd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:46.416480 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28ca97-335c-4cbb-919a-2fe8301addd9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:46.417540 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed28ca97-335c-4cbb-919a-2fe8301addd9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:46.419229 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed28ca97-335c-4cbb-919a-2fe8301addd9 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:46.430200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-ed28ca97-335c-4cbb-919a-2fe8301addd9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 04:09:46.430200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:46.430200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 04:09:46.430200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 04:09:46.430200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 04:09:46.430200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 04:09:46.430200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 04:09:46.430200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:46.430200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 04:09:46.430200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:46.430200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 04:09:46.430200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:46.430200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:46.430200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:46.430200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:46.430200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:46.430200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:46.431231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:46.431231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:46.431231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:46.431231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:46.431231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:46.431231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:46.431231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 04:09:46.431231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 04:09:46.431231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 04:09:46.431231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 04:09:46.431231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:46.431231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:46.431231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:46.431231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:46.431231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:46.431231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:46.432123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:46.432123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:46.432123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:46.432123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:46.432123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:46.432123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:46.432123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:46.432123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:46.432123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:46.432123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:46.432123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 04:09:46.432123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:46.432123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 04:09:46.432123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:46.435101 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 458/909] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 27 04:09:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:46.442098 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed28ca97-335c-4cbb-919a-2fe8301addd9 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:46.478318 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b66658d-6b3c-400b-a4ff-265cec819048 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:46.480142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b66658d-6b3c-400b-a4ff-265cec819048 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:46.488296 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:46.524383 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:46.580999 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed28ca97-335c-4cbb-919a-2fe8301addd9 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:46.602960 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 452/910] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:09:46 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 04:09:46.608304 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b66658d-6b3c-400b-a4ff-265cec819048 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:46.623588 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c4c98-4d75-4d53-8b53-14212b709df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:46.624535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c4c98-4d75-4d53-8b53-14212b709df7 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:46.625090 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c4c98-4d75-4d53-8b53-14212b709df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:46.625753 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c4c98-4d75-4d53-8b53-14212b709df7 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:46.626702 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68c4c98-4d75-4d53-8b53-14212b709df7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:46.627220 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68c4c98-4d75-4d53-8b53-14212b709df7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:46.628808 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68c4c98-4d75-4d53-8b53-14212b709df7 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:46.634325 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 459/911] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 27 04:09:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:46.639552 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c4c98-4d75-4d53-8b53-14212b709df7 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:46.732425 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b66658d-6b3c-400b-a4ff-265cec819048 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:46.749931 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:46.761659 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c4c98-4d75-4d53-8b53-14212b709df7 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:46.773505 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:46.781187 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb12737-c1cc-4819-9160-2fcab0c84178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:46.782430 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb12737-c1cc-4819-9160-2fcab0c84178 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:46.783115 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb12737-c1cc-4819-9160-2fcab0c84178 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:46.783407 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb12737-c1cc-4819-9160-2fcab0c84178 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:46.799621 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb12737-c1cc-4819-9160-2fcab0c84178 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:46.801071 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb12737-c1cc-4819-9160-2fcab0c84178 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:46.803669 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa89f8b-7a78-49a2-a758-d3886084be5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:46.804565 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa89f8b-7a78-49a2-a758-d3886084be5b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:46.805051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa89f8b-7a78-49a2-a758-d3886084be5b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:46.805529 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa89f8b-7a78-49a2-a758-d3886084be5b None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:46.806451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa89f8b-7a78-49a2-a758-d3886084be5b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:46.806981 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa89f8b-7a78-49a2-a758-d3886084be5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:46.808614 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa89f8b-7a78-49a2-a758-d3886084be5b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:46.815616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 460/912] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 27 04:09:46 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:46.816715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-5eb12737-c1cc-4819-9160-2fcab0c84178 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:46.816715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:46.816715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:46.816715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:46.816715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:46.816715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:46.816715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:46.816715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:46.816715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:46.816715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:46.816715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:46.816715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:46.816715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:46.816715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:46.816715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:46.817589 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:46.817589 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:46.817589 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:46.817589 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:46.817589 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:46.817589 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:46.817589 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:46.817589 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:46.817589 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:46.817589 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:46.817589 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:46.817589 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:46.817589 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:46.817589 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:46.817589 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:46.817589 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:46.817589 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:46.818551 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:46.818551 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:46.818551 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:46.818551 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:46.822565 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 453/913] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:46.824176 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa89f8b-7a78-49a2-a758-d3886084be5b None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:46.831756 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb12737-c1cc-4819-9160-2fcab0c84178 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:46.940954 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa89f8b-7a78-49a2-a758-d3886084be5b None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:46.957917 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb12737-c1cc-4819-9160-2fcab0c84178 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:46.980581 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99527e48-26d8-415c-bcd7-63588513aecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:46.981222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99527e48-26d8-415c-bcd7-63588513aecd None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:46.981509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99527e48-26d8-415c-bcd7-63588513aecd None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:46.981785 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99527e48-26d8-415c-bcd7-63588513aecd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:46.982613 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99527e48-26d8-415c-bcd7-63588513aecd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:46.983930 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99527e48-26d8-415c-bcd7-63588513aecd None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:47.001095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463fdd4-47e5-4b92-807f-0b5214e36f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:47.001725 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463fdd4-47e5-4b92-807f-0b5214e36f63 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:47.002030 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463fdd4-47e5-4b92-807f-0b5214e36f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:47.002309 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463fdd4-47e5-4b92-807f-0b5214e36f63 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:47.003122 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c463fdd4-47e5-4b92-807f-0b5214e36f63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:47.003433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c463fdd4-47e5-4b92-807f-0b5214e36f63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:47.004809 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c463fdd4-47e5-4b92-807f-0b5214e36f63 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:47.005783 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 461/914] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:09:46 2022] POST /identity/v3/endpoints => generated 346 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:09:47.024593 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 454/915] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:47.030475 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99527e48-26d8-415c-bcd7-63588513aecd None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:47.045471 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:47.163482 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99527e48-26d8-415c-bcd7-63588513aecd None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:47.200448 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650db6df-c272-4180-afb0-89213e6ca0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:47.201183 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650db6df-c272-4180-afb0-89213e6ca0df None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:47.201385 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650db6df-c272-4180-afb0-89213e6ca0df None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:47.201669 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650db6df-c272-4180-afb0-89213e6ca0df None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:47.209179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650db6df-c272-4180-afb0-89213e6ca0df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:47.210682 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650db6df-c272-4180-afb0-89213e6ca0df None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:47.222209 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-650db6df-c272-4180-afb0-89213e6ca0df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:47.222209 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:47.222209 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:47.222209 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:47.222209 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:47.222209 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:47.222209 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:47.222209 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:47.222209 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:47.222209 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:47.222209 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:47.222209 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:47.222209 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:09:47.222209 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:09:47.222209 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:09:47.223109 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:09:47.223109 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:47.223109 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:47.223109 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:47.223109 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:47.223109 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:47.223109 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:47.223109 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:47.223109 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:47.223109 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:47.223109 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:47.223109 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:47.223109 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:47.223109 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:47.223109 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:47.223109 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:47.223109 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:47.223982 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:09:47.223982 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:09:47.223982 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:09:47.223982 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:09:47.223982 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:09:47.223982 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:09:47.223982 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:47.223982 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:47.228120 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 462/916] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:47.229473 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463fdd4-47e5-4b92-807f-0b5214e36f63 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:47.279335 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:47.336583 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463fdd4-47e5-4b92-807f-0b5214e36f63 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:47.338084 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b60b06-84d7-497e-8a28-91fc658478d5 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:47.338375 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b60b06-84d7-497e-8a28-91fc658478d5 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:47.338649 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b60b06-84d7-497e-8a28-91fc658478d5 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:47.340162 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 463/917] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:47.346558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6515b45-079c-4c37-930a-113489af1ff7 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:47.346842 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6515b45-079c-4c37-930a-113489af1ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:47.347125 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6515b45-079c-4c37-930a-113489af1ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:47.370372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-e6515b45-079c-4c37-930a-113489af1ff7 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:47.372273 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e6515b45-079c-4c37-930a-113489af1ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:47.379696 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4397b4-926d-4d7f-8628-8d93862b4978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:47.380664 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4397b4-926d-4d7f-8628-8d93862b4978 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:47.381129 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4397b4-926d-4d7f-8628-8d93862b4978 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:47.381629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4397b4-926d-4d7f-8628-8d93862b4978 None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:47.386163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4397b4-926d-4d7f-8628-8d93862b4978 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:47.386163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4397b4-926d-4d7f-8628-8d93862b4978 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:47.386163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4397b4-926d-4d7f-8628-8d93862b4978 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:47.402167 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 455/918] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:09:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:47.405209 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4397b4-926d-4d7f-8628-8d93862b4978 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:47.439454 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:47.447689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 464/919] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 04:09:47.448993 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d91fc-b218-409c-893d-ac0a8783e245 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:47.449250 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d91fc-b218-409c-893d-ac0a8783e245 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:47.449523 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d91fc-b218-409c-893d-ac0a8783e245 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:47.451081 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 465/920] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:47.459946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd48023-c19f-475b-8bf1-91ca7e110792 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:47.460336 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd48023-c19f-475b-8bf1-91ca7e110792 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:47.460618 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd48023-c19f-475b-8bf1-91ca7e110792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:47.481828 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-bdd48023-c19f-475b-8bf1-91ca7e110792 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:47.483160 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-bdd48023-c19f-475b-8bf1-91ca7e110792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:47.533946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4397b4-926d-4d7f-8628-8d93862b4978 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:47.536782 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:47.557609 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 466/921] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:09:47.558747 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd48023-c19f-475b-8bf1-91ca7e110792 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:47.579665 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51df484-068e-4ba9-babd-581e82f09aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:47.580797 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51df484-068e-4ba9-babd-581e82f09aab None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:47.581090 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51df484-068e-4ba9-babd-581e82f09aab None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:47.581371 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51df484-068e-4ba9-babd-581e82f09aab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:47.582612 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51df484-068e-4ba9-babd-581e82f09aab None admin] RBAC: Adding query filter params (role.id=3298581080b940c1ab0a41e5bbc8953f, scope.project.id=e646d320aa5a4f78bfccaf941cf46478, user.id=8ec07ee1efba4e8089b94f24be338f86) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:47.582930 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51df484-068e-4ba9-babd-581e82f09aab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:47.584368 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51df484-068e-4ba9-babd-581e82f09aab None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:47.590283 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 456/922] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:09:47 2022] GET /identity/v3/role_assignments?user.id=8ec07ee1efba4e8089b94f24be338f86&scope.project.id=e646d320aa5a4f78bfccaf941cf46478&role.id=3298581080b940c1ab0a41e5bbc8953f => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:09:47.590887 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51df484-068e-4ba9-babd-581e82f09aab None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:47.706068 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd48023-c19f-475b-8bf1-91ca7e110792 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:47.718637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51df484-068e-4ba9-babd-581e82f09aab None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:47.743061 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:47.755947 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5c33d-0748-4947-b241-e401c7b2f173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:47.756962 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5c33d-0748-4947-b241-e401c7b2f173 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:47.757552 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5c33d-0748-4947-b241-e401c7b2f173 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:47.758065 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5c33d-0748-4947-b241-e401c7b2f173 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:47.759003 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5c33d-0748-4947-b241-e401c7b2f173 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:47.760589 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5c33d-0748-4947-b241-e401c7b2f173 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:47.763298 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508c5dc0-5023-40f9-a015-054cb3cf6cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:47.763946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c5dc0-5023-40f9-a015-054cb3cf6cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:47.764231 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c5dc0-5023-40f9-a015-054cb3cf6cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:47.764509 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c5dc0-5023-40f9-a015-054cb3cf6cf0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:47.765151 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508c5dc0-5023-40f9-a015-054cb3cf6cf0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:47.767103 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508c5dc0-5023-40f9-a015-054cb3cf6cf0 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:47.772850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-c5d5c33d-0748-4947-b241-e401c7b2f173 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 04:09:47.772850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:47.772850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 04:09:47.772850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 04:09:47.772850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 04:09:47.772850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 04:09:47.772850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 04:09:47.772850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:47.772850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 04:09:47.772850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:47.772850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 04:09:47.772850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:47.772850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:47.772850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:47.772850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:47.772850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:47.773767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:47.773767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:47.773767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:47.773767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:47.773767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:47.773767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:47.773767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:47.773767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 04:09:47.773767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 04:09:47.773767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 04:09:47.773767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 04:09:47.773767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:47.773767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:47.773767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:47.773767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:47.773767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:47.774641 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:47.774641 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:47.774641 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:47.774641 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:47.774641 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:47.774641 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:47.774641 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:47.774641 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:47.774641 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:47.774641 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:47.774641 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:47.774641 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 04:09:47.774641 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:47.774641 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 04:09:47.774641 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:47.776314 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-508c5dc0-5023-40f9-a015-054cb3cf6cf0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 04:09:47.776314 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:47.776314 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 04:09:47.776314 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 04:09:47.776314 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 04:09:47.776314 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 04:09:47.776314 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 04:09:47.776314 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:47.776314 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 04:09:47.776314 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:47.776314 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 04:09:47.776314 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:47.776314 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:47.776314 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:47.776314 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:47.776314 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:47.776314 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:47.777236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:47.777236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:47.777236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:47.777236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:47.777236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:47.777236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:47.777236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 04:09:47.777236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 04:09:47.777236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 04:09:47.777236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 04:09:47.777236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:47.777236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:47.777236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:47.777236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:47.777236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:47.777236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:47.778095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:47.778095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:47.778095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:47.778095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:47.778095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:47.778095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:47.778095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:47.778095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:47.778095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:47.778095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:47.778095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 04:09:47.778095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:47.778095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 04:09:47.778095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:47.778095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 457/923] 10.222.0.46 () {56 vars in 1126 bytes} [Wed Jul 27 04:09:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:47.780612 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 467/924] 10.222.0.46 () {56 vars in 1132 bytes} [Wed Jul 27 04:09:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:47.787099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508c5dc0-5023-40f9-a015-054cb3cf6cf0 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:47.788297 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5c33d-0748-4947-b241-e401c7b2f173 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:47.843332 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:47.896724 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5c33d-0748-4947-b241-e401c7b2f173 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:47.921467 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508c5dc0-5023-40f9-a015-054cb3cf6cf0 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:47.935220 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0c9c7-c7d6-487b-812e-90e2e439167f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:47.935849 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0c9c7-c7d6-487b-812e-90e2e439167f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:47.936131 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0c9c7-c7d6-487b-812e-90e2e439167f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:47.936408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0c9c7-c7d6-487b-812e-90e2e439167f None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:47.937095 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c0c9c7-c7d6-487b-812e-90e2e439167f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:47.937405 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c0c9c7-c7d6-487b-812e-90e2e439167f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:47.940122 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c0c9c7-c7d6-487b-812e-90e2e439167f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:47.949727 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 468/925] 10.222.0.46 () {56 vars in 1126 bytes} [Wed Jul 27 04:09:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:47.950791 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0c9c7-c7d6-487b-812e-90e2e439167f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:47.966876 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef31a9e-e213-4986-ac74-2f8070271e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:47.967343 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef31a9e-e213-4986-ac74-2f8070271e6a None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:47.967624 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef31a9e-e213-4986-ac74-2f8070271e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:47.967896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef31a9e-e213-4986-ac74-2f8070271e6a None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:47.968583 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef31a9e-e213-4986-ac74-2f8070271e6a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:47.969457 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef31a9e-e213-4986-ac74-2f8070271e6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:47.971509 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef31a9e-e213-4986-ac74-2f8070271e6a None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:47.978596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 458/926] 10.222.0.46 () {56 vars in 1122 bytes} [Wed Jul 27 04:09:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:47.983774 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef31a9e-e213-4986-ac74-2f8070271e6a None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:48.074426 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:48.086749 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0c9c7-c7d6-487b-812e-90e2e439167f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:48.098366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:48.107576 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef31a9e-e213-4986-ac74-2f8070271e6a None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:48.122609 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac51a4-95d6-4acc-b376-57de830a6762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:48.123234 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac51a4-95d6-4acc-b376-57de830a6762 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:48.123515 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac51a4-95d6-4acc-b376-57de830a6762 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:48.123796 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac51a4-95d6-4acc-b376-57de830a6762 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:48.124477 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac51a4-95d6-4acc-b376-57de830a6762 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:48.124778 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac51a4-95d6-4acc-b376-57de830a6762 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:48.126263 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac51a4-95d6-4acc-b376-57de830a6762 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:48.135767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 469/927] 10.222.0.46 () {56 vars in 1126 bytes} [Wed Jul 27 04:09:47 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:48.136989 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac51a4-95d6-4acc-b376-57de830a6762 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:48.158976 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cebd45-1b77-406d-9a9c-02d941fa49ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:48.159545 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cebd45-1b77-406d-9a9c-02d941fa49ea None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:48.159823 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cebd45-1b77-406d-9a9c-02d941fa49ea None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:48.160095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cebd45-1b77-406d-9a9c-02d941fa49ea None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:48.160848 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cebd45-1b77-406d-9a9c-02d941fa49ea None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:48.161070 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cebd45-1b77-406d-9a9c-02d941fa49ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:48.162489 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cebd45-1b77-406d-9a9c-02d941fa49ea None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:48.173469 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 459/928] 10.222.0.46 () {56 vars in 1122 bytes} [Wed Jul 27 04:09:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:48.174468 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cebd45-1b77-406d-9a9c-02d941fa49ea None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:48.272455 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac51a4-95d6-4acc-b376-57de830a6762 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:48.285219 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cebd45-1b77-406d-9a9c-02d941fa49ea None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:48.296488 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:48.325629 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1da0e-37e4-4012-ac43-5d11ae949ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:48.327527 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1da0e-37e4-4012-ac43-5d11ae949ec4 None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:48.328705 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1da0e-37e4-4012-ac43-5d11ae949ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:48.329287 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1da0e-37e4-4012-ac43-5d11ae949ec4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:48.330403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a1da0e-37e4-4012-ac43-5d11ae949ec4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:48.332178 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d5ce8e-5265-48b7-af4b-12dd850b5706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:48.332424 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a1da0e-37e4-4012-ac43-5d11ae949ec4 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:48.333015 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5ce8e-5265-48b7-af4b-12dd850b5706 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:48.334490 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5ce8e-5265-48b7-af4b-12dd850b5706 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:48.334861 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5ce8e-5265-48b7-af4b-12dd850b5706 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:48.335736 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d5ce8e-5265-48b7-af4b-12dd850b5706 None admin] RBAC: Adding query filter params (interface=public, service_id=b24e3edeb77c4c6cb924f9709225e28b, region_id=RegionOne) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:48.336145 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d5ce8e-5265-48b7-af4b-12dd850b5706 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:48.337615 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d5ce8e-5265-48b7-af4b-12dd850b5706 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:48.348982 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 460/929] 10.222.0.46 () {56 vars in 1251 bytes} [Wed Jul 27 04:09:48 2022] GET /identity/v3/endpoints?service_id=b24e3edeb77c4c6cb924f9709225e28b&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:09:48.354014 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d5ce8e-5265-48b7-af4b-12dd850b5706 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:48.372454 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 470/930] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:09:48 2022] POST /identity/v3/endpoints => generated 347 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:09:48.411291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:48.504937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d5ce8e-5265-48b7-af4b-12dd850b5706 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:48.552586 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0492beb6-54a9-4f49-8208-c00c3e82bdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:48.553180 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492beb6-54a9-4f49-8208-c00c3e82bdf4 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:48.553468 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492beb6-54a9-4f49-8208-c00c3e82bdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:48.553743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492beb6-54a9-4f49-8208-c00c3e82bdf4 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:48.554436 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0492beb6-54a9-4f49-8208-c00c3e82bdf4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:48.555874 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0492beb6-54a9-4f49-8208-c00c3e82bdf4 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:48.562260 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 461/931] 10.222.0.46 () {56 vars in 1087 bytes} [Wed Jul 27 04:09:48 2022] GET /identity/v3/services => generated 1497 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:09:49.188583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a31ac-cc07-41d7-828c-94b0d8bbe642 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:49.188988 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a31ac-cc07-41d7-828c-94b0d8bbe642 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:49.189117 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a31ac-cc07-41d7-828c-94b0d8bbe642 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:49.190806 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 471/932] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:49.196268 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab405cf-97fa-4de7-90b6-7695e702cc7b None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:49.196551 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab405cf-97fa-4de7-90b6-7695e702cc7b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:49.196825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab405cf-97fa-4de7-90b6-7695e702cc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:49.215549 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-cab405cf-97fa-4de7-90b6-7695e702cc7b None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:49.216720 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-cab405cf-97fa-4de7-90b6-7695e702cc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:49.287582 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 462/933] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:09:49.309648 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a31ac-cc07-41d7-828c-94b0d8bbe642 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:49.415991 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a31ac-cc07-41d7-828c-94b0d8bbe642 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:49.450374 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b691855d-920e-4301-8864-a20aa377358d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:49.454306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691855d-920e-4301-8864-a20aa377358d None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:49.454306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691855d-920e-4301-8864-a20aa377358d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:49.454306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691855d-920e-4301-8864-a20aa377358d None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:49.454306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b691855d-920e-4301-8864-a20aa377358d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:49.454306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b691855d-920e-4301-8864-a20aa377358d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:49.457158 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: WARNING keystone.server.flask.common [None req-b691855d-920e-4301-8864-a20aa377358d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 27 04:09:49.477021 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.sql.core [None req-b691855d-920e-4301-8864-a20aa377358d None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 27 04:09:49.477021 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 27 04:09:49.477021 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [parameters: {'id': 'b25d522c5642450ea21aeab803a7a25f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 27 04:09:49.477021 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80239) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 27 04:09:49.486012 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-b691855d-920e-4301-8864-a20aa377358d None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b25d522c5642450ea21aeab803a7a25f.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b25d522c5642450ea21aeab803a7a25f. Jul 27 04:09:49.486012 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:49.486012 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 04:09:49.486012 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 04:09:49.486012 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 04:09:49.486012 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 04:09:49.486012 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 04:09:49.486012 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 04:09:49.486012 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 04:09:49.486012 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application conn.query(q) Jul 27 04:09:49.486012 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 04:09:49.486012 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 04:09:49.486012 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 04:09:49.487037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application result.read() Jul 27 04:09:49.487037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 04:09:49.487037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 04:09:49.487037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 04:09:49.487037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 04:09:49.487037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 04:09:49.487037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 04:09:49.487037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 04:09:49.487037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 04:09:49.487037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 27 04:09:49.487037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:49.487037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 27 04:09:49.487037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:49.487037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:49.487037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 27 04:09:49.487037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 27 04:09:49.487037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 27 04:09:49.487037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 27 04:09:49.488385 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 04:09:49.488385 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application next(self.gen) Jul 27 04:09:49.488385 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 27 04:09:49.488385 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application yield resource Jul 27 04:09:49.488385 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 04:09:49.488385 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application next(self.gen) Jul 27 04:09:49.488385 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 27 04:09:49.488385 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self.session.rollback() Jul 27 04:09:49.488385 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 27 04:09:49.488385 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self.force_reraise() Jul 27 04:09:49.488385 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 27 04:09:49.488385 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise self.value Jul 27 04:09:49.488385 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 27 04:09:49.488385 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 27 04:09:49.488385 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 27 04:09:49.488385 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application session.commit() Jul 27 04:09:49.488385 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 27 04:09:49.489279 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 27 04:09:49.489279 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 27 04:09:49.489279 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._prepare_impl() Jul 27 04:09:49.489279 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 27 04:09:49.489279 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self.session.flush() Jul 27 04:09:49.489279 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 27 04:09:49.489279 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._flush(objects) Jul 27 04:09:49.489279 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 27 04:09:49.489279 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 27 04:09:49.489279 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 27 04:09:49.489279 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application compat.raise_( Jul 27 04:09:49.489279 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 04:09:49.489279 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception Jul 27 04:09:49.489279 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 27 04:09:49.489279 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application flush_context.execute() Jul 27 04:09:49.489279 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 27 04:09:49.489279 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rec.execute(self) Jul 27 04:09:49.490711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 27 04:09:49.490711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 27 04:09:49.490711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 27 04:09:49.490711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 27 04:09:49.490711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 27 04:09:49.490711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 27 04:09:49.490711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 27 04:09:49.490711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 27 04:09:49.490711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 27 04:09:49.490711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 27 04:09:49.490711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 27 04:09:49.490711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 27 04:09:49.490711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 27 04:09:49.490711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 27 04:09:49.490711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 27 04:09:49.492232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 27 04:09:49.492232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 04:09:49.492232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception Jul 27 04:09:49.492232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 04:09:49.492232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 04:09:49.492232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 04:09:49.492232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 04:09:49.492232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 04:09:49.492232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 04:09:49.492232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 04:09:49.492232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application conn.query(q) Jul 27 04:09:49.492232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 04:09:49.492232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 04:09:49.492232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 04:09:49.492232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application result.read() Jul 27 04:09:49.492232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 04:09:49.492232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 04:09:49.493262 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 04:09:49.493262 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 04:09:49.493262 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 04:09:49.493262 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 04:09:49.493262 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 04:09:49.493262 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 04:09:49.493262 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 27 04:09:49.493262 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 27 04:09:49.493262 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [parameters: {'id': 'b25d522c5642450ea21aeab803a7a25f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 27 04:09:49.493262 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 27 04:09:49.493262 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:49.493262 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 04:09:49.493262 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:49.493262 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:49.493262 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 27 04:09:49.499341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 27 04:09:49.499341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 27 04:09:49.499341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 27 04:09:49.499341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 27 04:09:49.499341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:49.499341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 04:09:49.499341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:49.499341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:49.499341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:49.499341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:49.499341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:49.499341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:49.499341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:49.499341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:49.499341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:49.499341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:49.499341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:49.502499 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:49.502499 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 27 04:09:49.502499 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 27 04:09:49.502499 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:49.502499 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:49.502499 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 27 04:09:49.502499 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 27 04:09:49.502499 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b25d522c5642450ea21aeab803a7a25f. Jul 27 04:09:49.502499 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:49.502499 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 472/934] 10.222.0.46 () {60 vars in 1140 bytes} [Wed Jul 27 04:09:49 2022] POST /identity/v3/projects => generated 262 bytes in 178 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:49.503801 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab405cf-97fa-4de7-90b6-7695e702cc7b None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:49.511635 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:49.604009 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab405cf-97fa-4de7-90b6-7695e702cc7b None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:49.640502 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4eaf14-4d2c-4bee-9fff-fffe6afa2c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:49.642254 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4eaf14-4d2c-4bee-9fff-fffe6afa2c59 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:49.642781 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4eaf14-4d2c-4bee-9fff-fffe6afa2c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:49.643270 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4eaf14-4d2c-4bee-9fff-fffe6afa2c59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:49.650822 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4eaf14-4d2c-4bee-9fff-fffe6afa2c59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:49.652644 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4eaf14-4d2c-4bee-9fff-fffe6afa2c59 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:49.666039 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-dd4eaf14-4d2c-4bee-9fff-fffe6afa2c59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:49.666039 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:49.666039 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:49.666039 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:49.666039 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:49.666039 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:49.666039 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:49.666039 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:49.666039 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:49.666039 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:49.666039 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:49.666039 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:49.666039 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:09:49.666039 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:09:49.666039 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:09:49.667023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:09:49.667023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:49.667023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:49.667023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:49.667023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:49.667023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:49.667023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:49.667023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:49.667023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:49.667023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:49.667023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:49.667023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:49.667023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:49.667023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:49.667023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:49.667023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:49.667023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:49.667994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:09:49.667994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:09:49.667994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:09:49.667994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:09:49.667994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:09:49.667994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:09:49.667994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:49.667994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:49.670591 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 463/935] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:49.676526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b691855d-920e-4301-8864-a20aa377358d None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:49.703674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b7cf3-3d45-4f79-9faf-489bcf2f7ede None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:49.703860 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b7cf3-3d45-4f79-9faf-489bcf2f7ede None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:49.704171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b7cf3-3d45-4f79-9faf-489bcf2f7ede None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:49.705870 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 464/936] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 04:09:49.710791 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f7be7-ce3e-4572-a560-ad25e7078eda None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:49.711074 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f7be7-ce3e-4572-a560-ad25e7078eda None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:49.711416 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f7be7-ce3e-4572-a560-ad25e7078eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:49.726534 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:49.736517 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-fd1f7be7-ce3e-4572-a560-ad25e7078eda None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:49.737896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-fd1f7be7-ce3e-4572-a560-ad25e7078eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:49.815177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 465/937] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 04:09:49.823292 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b691855d-920e-4301-8864-a20aa377358d None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:49.851581 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f7be7-ce3e-4572-a560-ad25e7078eda None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:49.854786 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:49.863100 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43147a-2acc-4311-b5ea-79149c8a7293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:49.863727 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43147a-2acc-4311-b5ea-79149c8a7293 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:49.864014 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43147a-2acc-4311-b5ea-79149c8a7293 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:49.864294 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43147a-2acc-4311-b5ea-79149c8a7293 None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:49.865028 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed43147a-2acc-4311-b5ea-79149c8a7293 None admin] RBAC: Adding query filter params (name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:49.865343 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed43147a-2acc-4311-b5ea-79149c8a7293 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:49.867136 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed43147a-2acc-4311-b5ea-79149c8a7293 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:49.894001 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 473/938] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Jul 27 04:09:49 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:49.970228 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f7be7-ce3e-4572-a560-ad25e7078eda None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:50.012063 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00552b43-f246-426e-9dde-78e19f826425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:50.012736 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00552b43-f246-426e-9dde-78e19f826425 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:50.013019 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00552b43-f246-426e-9dde-78e19f826425 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:50.013296 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00552b43-f246-426e-9dde-78e19f826425 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:50.025633 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00552b43-f246-426e-9dde-78e19f826425 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:09:50.030641 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-00552b43-f246-426e-9dde-78e19f826425 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 04:09:50.030641 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:50.030641 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:50.030641 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:50.030641 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:50.030641 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:50.030641 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:50.030641 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:50.030641 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:50.030641 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:50.030641 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:50.030641 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:50.030641 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:09:50.030641 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:09:50.030641 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:09:50.030641 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:09:50.031711 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:09:50.031711 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:09:50.031711 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:09:50.031711 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:09:50.031711 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:50.031711 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:50.031711 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:50.031711 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:50.031711 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:50.031711 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:50.031711 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:50.031711 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:50.031711 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:50.031711 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:50.031711 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:50.031711 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:50.032762 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:50.032762 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:50.032762 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:50.032762 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:50.032762 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:09:50.032762 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:09:50.032762 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:09:50.032762 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:09:50.032762 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:09:50.032762 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:09:50.032762 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 04:09:50.032762 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:50.032762 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 466/939] 10.222.0.46 () {56 vars in 1120 bytes} [Wed Jul 27 04:09:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:09:50.039978 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43147a-2acc-4311-b5ea-79149c8a7293 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:50.141218 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:50.152704 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43147a-2acc-4311-b5ea-79149c8a7293 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:50.189438 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8722a0e-010a-453c-b31d-3d7fe32c5140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:50.190511 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8722a0e-010a-453c-b31d-3d7fe32c5140 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:50.191055 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8722a0e-010a-453c-b31d-3d7fe32c5140 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:50.191539 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8722a0e-010a-453c-b31d-3d7fe32c5140 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:50.192457 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8722a0e-010a-453c-b31d-3d7fe32c5140 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:50.192959 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8722a0e-010a-453c-b31d-3d7fe32c5140 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:50.194574 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8722a0e-010a-453c-b31d-3d7fe32c5140 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:50.203659 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 474/940] 10.222.0.46 () {56 vars in 1115 bytes} [Wed Jul 27 04:09:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:50.209556 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00552b43-f246-426e-9dde-78e19f826425 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:50.279770 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:50.320431 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00552b43-f246-426e-9dde-78e19f826425 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:50.359945 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6f8cb-8c99-4014-9d0b-f141dbc894c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:50.361015 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6f8cb-8c99-4014-9d0b-f141dbc894c1 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:50.361308 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6f8cb-8c99-4014-9d0b-f141dbc894c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:50.361584 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6f8cb-8c99-4014-9d0b-f141dbc894c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:50.374269 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b6f8cb-8c99-4014-9d0b-f141dbc894c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:50.376139 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b6f8cb-8c99-4014-9d0b-f141dbc894c1 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:50.391508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-c2b6f8cb-8c99-4014-9d0b-f141dbc894c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:50.391508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:50.391508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:50.391508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:50.391508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:50.391508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:50.391508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:50.391508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:50.391508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:50.391508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:50.391508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:50.391508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:50.391508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:50.391508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:50.391508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:50.392409 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:50.392409 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:50.392409 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:50.392409 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:50.392409 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:50.392409 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:50.392409 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:50.392409 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:50.392409 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:50.392409 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:50.392409 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:50.392409 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:50.392409 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:50.392409 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:50.392409 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:50.392409 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:50.392409 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:50.393286 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:50.393286 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:50.393286 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:50.393286 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:50.393286 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 467/941] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:50.399741 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8722a0e-010a-453c-b31d-3d7fe32c5140 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:50.523093 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8722a0e-010a-453c-b31d-3d7fe32c5140 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:50.556795 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b14d7-3d25-4241-afa5-cc49c38a6ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:50.557344 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b14d7-3d25-4241-afa5-cc49c38a6ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:50.557625 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b14d7-3d25-4241-afa5-cc49c38a6ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:50.557903 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b14d7-3d25-4241-afa5-cc49c38a6ab6 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:50.558684 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b14d7-3d25-4241-afa5-cc49c38a6ab6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:50.558991 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b14d7-3d25-4241-afa5-cc49c38a6ab6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:50.560355 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b14d7-3d25-4241-afa5-cc49c38a6ab6 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:50.577036 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 475/942] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:50.583337 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6f8cb-8c99-4014-9d0b-f141dbc894c1 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:50.633149 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f61e8-186e-4c55-8def-778d1ce647fb None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:50.633796 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f61e8-186e-4c55-8def-778d1ce647fb None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:50.634320 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f61e8-186e-4c55-8def-778d1ce647fb None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:50.636113 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 476/943] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 04:09:50.641285 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0054e-7db4-4c61-a439-8b9221249615 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:50.641850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0054e-7db4-4c61-a439-8b9221249615 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:50.642383 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0054e-7db4-4c61-a439-8b9221249615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:50.661374 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-4da0054e-7db4-4c61-a439-8b9221249615 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:50.663291 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-4da0054e-7db4-4c61-a439-8b9221249615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:50.671445 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:50.696560 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6f8cb-8c99-4014-9d0b-f141dbc894c1 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:50.714955 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:50.737803 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ba75b-b99e-4d95-aef5-4b6b030e66f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:50.738500 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ba75b-b99e-4d95-aef5-4b6b030e66f8 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:50.738850 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ba75b-b99e-4d95-aef5-4b6b030e66f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:50.739131 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ba75b-b99e-4d95-aef5-4b6b030e66f8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:50.743189 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 477/944] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 04:09:50.747388 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67470c-f6ea-4180-aa03-16cd8399b3b0 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:50.747752 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67470c-f6ea-4180-aa03-16cd8399b3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:50.748108 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67470c-f6ea-4180-aa03-16cd8399b3b0 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:50.749482 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 478/945] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:50.753857 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ba75b-b99e-4d95-aef5-4b6b030e66f8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:50.753857 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ba75b-b99e-4d95-aef5-4b6b030e66f8 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:50.755426 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998d562-ecc3-4ffd-a4bb-01a0264e3f75 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:50.755945 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998d562-ecc3-4ffd-a4bb-01a0264e3f75 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:50.756230 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998d562-ecc3-4ffd-a4bb-01a0264e3f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:50.766242 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-c13ba75b-b99e-4d95-aef5-4b6b030e66f8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 04:09:50.766242 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:50.766242 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:50.766242 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:50.766242 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:50.766242 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:50.766242 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:50.766242 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:50.766242 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:50.766242 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:50.766242 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:50.766242 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:50.766242 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:09:50.766242 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:09:50.766242 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:09:50.766242 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:09:50.767219 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:50.767219 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:50.767219 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:09:50.767219 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:50.767219 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:09:50.767219 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:50.767219 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:50.767219 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:50.767219 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:50.767219 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:50.767219 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:50.767219 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:50.767219 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:50.767219 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:50.767219 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:50.767219 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:50.767219 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:50.767219 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:50.768167 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:50.768167 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:50.768167 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:09:50.768167 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:09:50.768167 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:09:50.768167 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:09:50.768167 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:09:50.768167 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:09:50.768167 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 04:09:50.768167 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:50.771381 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 468/946] 10.222.0.46 () {56 vars in 1117 bytes} [Wed Jul 27 04:09:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:50.772221 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ba75b-b99e-4d95-aef5-4b6b030e66f8 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:50.776492 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-a998d562-ecc3-4ffd-a4bb-01a0264e3f75 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:50.777678 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a998d562-ecc3-4ffd-a4bb-01a0264e3f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:50.850899 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 479/947] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:09:50.851669 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a998d562-ecc3-4ffd-a4bb-01a0264e3f75 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:50.891769 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ba75b-b99e-4d95-aef5-4b6b030e66f8 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:50.923955 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:50.935091 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8d7f1-80c2-466b-bdaf-c512194cb655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:50.935747 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8d7f1-80c2-466b-bdaf-c512194cb655 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:50.936179 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8d7f1-80c2-466b-bdaf-c512194cb655 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:50.936469 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8d7f1-80c2-466b-bdaf-c512194cb655 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:50.938784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d8d7f1-80c2-466b-bdaf-c512194cb655 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:50.940452 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d8d7f1-80c2-466b-bdaf-c512194cb655 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:50.949772 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-77d8d7f1-80c2-466b-bdaf-c512194cb655 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 04:09:50.949772 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:50.949772 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 04:09:50.949772 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 04:09:50.949772 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 04:09:50.949772 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 04:09:50.949772 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 04:09:50.949772 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:50.949772 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 04:09:50.949772 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:50.949772 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 04:09:50.949772 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:50.949772 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:50.949772 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:50.949772 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:50.949772 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:50.949772 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:50.950816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:50.950816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:50.950816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:50.950816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:50.950816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:50.950816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:50.950816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 04:09:50.950816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 04:09:50.950816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 04:09:50.950816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 04:09:50.950816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:50.950816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:50.950816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:50.950816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:50.950816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:50.950816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:50.951653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:50.951653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:50.951653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:50.951653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:50.951653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:50.951653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:50.951653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:50.951653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:50.951653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:50.951653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:50.951653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 04:09:50.951653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:50.951653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 04:09:50.951653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:50.951653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 469/948] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:50.952450 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8d7f1-80c2-466b-bdaf-c512194cb655 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:50.961794 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a998d562-ecc3-4ffd-a4bb-01a0264e3f75 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:50.997297 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6571e2-78cc-4ef6-9634-94f6397dc297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:50.997929 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6571e2-78cc-4ef6-9634-94f6397dc297 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:50.998240 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6571e2-78cc-4ef6-9634-94f6397dc297 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:50.998356 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:50.998522 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6571e2-78cc-4ef6-9634-94f6397dc297 None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:50.999694 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6571e2-78cc-4ef6-9634-94f6397dc297 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:51.000665 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6571e2-78cc-4ef6-9634-94f6397dc297 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:51.002781 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6571e2-78cc-4ef6-9634-94f6397dc297 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:51.021507 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 480/949] 10.222.0.46 () {56 vars in 1149 bytes} [Wed Jul 27 04:09:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:51.026963 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6571e2-78cc-4ef6-9634-94f6397dc297 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:51.064025 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8d7f1-80c2-466b-bdaf-c512194cb655 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:51.100684 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6fbcf2-9c4f-4b61-8db7-edf2efc80018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:51.101384 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6fbcf2-9c4f-4b61-8db7-edf2efc80018 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:51.101614 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6fbcf2-9c4f-4b61-8db7-edf2efc80018 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:51.101887 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6fbcf2-9c4f-4b61-8db7-edf2efc80018 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:51.102569 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6fbcf2-9c4f-4b61-8db7-edf2efc80018 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:51.103931 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6fbcf2-9c4f-4b61-8db7-edf2efc80018 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:51.113529 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-1b6fbcf2-9c4f-4b61-8db7-edf2efc80018 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 04:09:51.113529 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:51.113529 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 04:09:51.113529 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 04:09:51.113529 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 04:09:51.113529 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 04:09:51.113529 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 04:09:51.113529 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:51.113529 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 04:09:51.113529 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:51.113529 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 04:09:51.113529 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:51.113529 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:51.113529 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:51.113529 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:51.113529 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:51.113529 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:51.114543 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:51.114543 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:51.114543 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:51.114543 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:51.114543 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:51.114543 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:51.114543 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 04:09:51.114543 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 04:09:51.114543 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 04:09:51.114543 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 04:09:51.114543 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:51.114543 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:51.114543 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:51.114543 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:51.114543 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:51.114543 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:51.115398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:51.115398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:51.115398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:51.115398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:51.115398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:51.115398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:51.115398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:51.115398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:51.115398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:51.115398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:51.115398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 04:09:51.115398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:51.115398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 04:09:51.115398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:51.120131 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 470/950] 10.222.0.46 () {56 vars in 1126 bytes} [Wed Jul 27 04:09:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:51.121019 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6fbcf2-9c4f-4b61-8db7-edf2efc80018 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:51.133681 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6571e2-78cc-4ef6-9634-94f6397dc297 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:51.173640 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:51.181233 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683aa6cb-c6cb-419c-9eea-22932c332998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:51.182095 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683aa6cb-c6cb-419c-9eea-22932c332998 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:51.182095 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683aa6cb-c6cb-419c-9eea-22932c332998 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:51.182385 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683aa6cb-c6cb-419c-9eea-22932c332998 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:51.183086 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683aa6cb-c6cb-419c-9eea-22932c332998 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:51.183329 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683aa6cb-c6cb-419c-9eea-22932c332998 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:51.184711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683aa6cb-c6cb-419c-9eea-22932c332998 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:51.195233 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 481/951] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Jul 27 04:09:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:51.195233 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683aa6cb-c6cb-419c-9eea-22932c332998 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:51.231560 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6fbcf2-9c4f-4b61-8db7-edf2efc80018 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:51.235331 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:51.268823 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7fd49a-467d-4f3b-87c0-08cfdf199e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:51.269461 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7fd49a-467d-4f3b-87c0-08cfdf199e25 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:51.269739 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7fd49a-467d-4f3b-87c0-08cfdf199e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:51.270041 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7fd49a-467d-4f3b-87c0-08cfdf199e25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:51.281797 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7fd49a-467d-4f3b-87c0-08cfdf199e25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:51.283433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7fd49a-467d-4f3b-87c0-08cfdf199e25 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:51.298902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-7e7fd49a-467d-4f3b-87c0-08cfdf199e25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:51.298902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:51.298902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:51.298902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:51.298902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:51.298902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:51.298902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:51.298902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:51.298902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:51.298902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:51.298902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:51.298902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:51.298902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:51.298902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:51.298902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:51.299799 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:51.299799 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:51.299799 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:51.299799 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:51.299799 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:51.299799 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:51.299799 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:51.299799 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:51.299799 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:51.299799 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:51.299799 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:51.299799 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:51.299799 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:51.299799 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:51.299799 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:51.299799 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:51.299799 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:51.300695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:51.300695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:51.300695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:51.300695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:51.300695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 471/952] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:51.300997 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7fd49a-467d-4f3b-87c0-08cfdf199e25 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:51.307302 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683aa6cb-c6cb-419c-9eea-22932c332998 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:51.341906 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b25857-c98e-4a05-8fd9-ae7053d48e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:51.342558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b25857-c98e-4a05-8fd9-ae7053d48e61 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:51.342840 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b25857-c98e-4a05-8fd9-ae7053d48e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:51.343115 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b25857-c98e-4a05-8fd9-ae7053d48e61 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:51.345050 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b25857-c98e-4a05-8fd9-ae7053d48e61 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:51.345050 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b25857-c98e-4a05-8fd9-ae7053d48e61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:51.346268 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b25857-c98e-4a05-8fd9-ae7053d48e61 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:51.355948 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 482/953] 10.222.0.46 () {56 vars in 1122 bytes} [Wed Jul 27 04:09:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:51.355948 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b25857-c98e-4a05-8fd9-ae7053d48e61 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:51.412913 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:51.415646 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7fd49a-467d-4f3b-87c0-08cfdf199e25 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:51.456147 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f4b6f-a470-4c8e-99d7-61a32e6d22de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:51.456493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f4b6f-a470-4c8e-99d7-61a32e6d22de None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:51.456789 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f4b6f-a470-4c8e-99d7-61a32e6d22de None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:51.457172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f4b6f-a470-4c8e-99d7-61a32e6d22de None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:51.457721 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f4b6f-a470-4c8e-99d7-61a32e6d22de None admin] RBAC: Adding query filter params (type=compute) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:51.458044 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f4b6f-a470-4c8e-99d7-61a32e6d22de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:51.459404 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f4b6f-a470-4c8e-99d7-61a32e6d22de None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:51.474763 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 472/954] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Jul 27 04:09:51 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:51.474763 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f4b6f-a470-4c8e-99d7-61a32e6d22de None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:51.475060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b25857-c98e-4a05-8fd9-ae7053d48e61 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:51.505107 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:51.515254 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c226a9-4800-40f0-b5b7-d7ed3fc93efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:51.516193 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c226a9-4800-40f0-b5b7-d7ed3fc93efd None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:51.516477 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c226a9-4800-40f0-b5b7-d7ed3fc93efd None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:51.516752 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c226a9-4800-40f0-b5b7-d7ed3fc93efd None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:51.517631 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c226a9-4800-40f0-b5b7-d7ed3fc93efd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:51.517937 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c226a9-4800-40f0-b5b7-d7ed3fc93efd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:51.519968 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c226a9-4800-40f0-b5b7-d7ed3fc93efd None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:51.536220 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 483/955] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:51.541914 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c226a9-4800-40f0-b5b7-d7ed3fc93efd None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:51.603635 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f4b6f-a470-4c8e-99d7-61a32e6d22de None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:51.642125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131cd9e3-62ae-48c5-bbd0-2cf894238587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:51.642678 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131cd9e3-62ae-48c5-bbd0-2cf894238587 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:51.642958 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131cd9e3-62ae-48c5-bbd0-2cf894238587 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:51.643232 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131cd9e3-62ae-48c5-bbd0-2cf894238587 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:51.644227 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131cd9e3-62ae-48c5-bbd0-2cf894238587 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:51.644536 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131cd9e3-62ae-48c5-bbd0-2cf894238587 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:51.645914 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131cd9e3-62ae-48c5-bbd0-2cf894238587 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:51.651854 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 473/956] 10.222.0.46 () {56 vars in 1122 bytes} [Wed Jul 27 04:09:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:51.657903 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131cd9e3-62ae-48c5-bbd0-2cf894238587 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:51.661728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c226a9-4800-40f0-b5b7-d7ed3fc93efd None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:51.690128 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:51.714551 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58660052-beac-4df5-bca2-d40b4a3dc33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:51.715503 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58660052-beac-4df5-bca2-d40b4a3dc33d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:51.716621 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58660052-beac-4df5-bca2-d40b4a3dc33d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:51.717094 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58660052-beac-4df5-bca2-d40b4a3dc33d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:51.727711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58660052-beac-4df5-bca2-d40b4a3dc33d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:51.729952 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58660052-beac-4df5-bca2-d40b4a3dc33d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:51.746958 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-58660052-beac-4df5-bca2-d40b4a3dc33d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:51.746958 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:51.746958 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:51.746958 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:51.746958 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:51.746958 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:51.746958 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:51.746958 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:51.746958 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:51.746958 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:51.746958 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:51.746958 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:51.746958 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:09:51.746958 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:09:51.746958 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:09:51.748006 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:09:51.748006 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:51.748006 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:51.748006 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:51.748006 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:51.748006 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:51.748006 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:51.748006 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:51.748006 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:51.748006 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:51.748006 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:51.748006 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:51.748006 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:51.748006 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:51.748006 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:51.748006 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:51.748006 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:51.748960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:09:51.748960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:09:51.748960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:09:51.748960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:09:51.748960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:09:51.748960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:09:51.748960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:51.748960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:51.752375 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 484/957] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:51.757882 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:51.762997 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58660052-beac-4df5-bca2-d40b4a3dc33d None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:51.774540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131cd9e3-62ae-48c5-bbd0-2cf894238587 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:51.811607 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445d413-fc89-40b0-bec1-d2f5016963c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:51.812264 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445d413-fc89-40b0-bec1-d2f5016963c2 None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:51.813089 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445d413-fc89-40b0-bec1-d2f5016963c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:51.813618 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445d413-fc89-40b0-bec1-d2f5016963c2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:51.814480 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2445d413-fc89-40b0-bec1-d2f5016963c2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:51.815857 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2445d413-fc89-40b0-bec1-d2f5016963c2 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:51.850761 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 474/958] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:09:51 2022] POST /identity/v3/endpoints => generated 349 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:09:51.893147 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58660052-beac-4df5-bca2-d40b4a3dc33d None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:51.943040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edddf63e-8bff-4f1b-911b-e6891019098e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:51.944087 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edddf63e-8bff-4f1b-911b-e6891019098e None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:51.944587 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edddf63e-8bff-4f1b-911b-e6891019098e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:51.945073 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edddf63e-8bff-4f1b-911b-e6891019098e None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:51.946168 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edddf63e-8bff-4f1b-911b-e6891019098e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:51.946700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edddf63e-8bff-4f1b-911b-e6891019098e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:51.948517 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edddf63e-8bff-4f1b-911b-e6891019098e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:51.969713 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 485/959] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:09:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:51.976002 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445d413-fc89-40b0-bec1-d2f5016963c2 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:52.090585 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445d413-fc89-40b0-bec1-d2f5016963c2 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:52.093268 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323010d-1362-4d03-9632-e3a6a2fc53da None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:52.093555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323010d-1362-4d03-9632-e3a6a2fc53da None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:52.094506 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323010d-1362-4d03-9632-e3a6a2fc53da None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:52.096041 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 486/960] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:52.101924 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca708439-9d69-4ad1-9cb2-cad1a02d2d22 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:52.102241 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca708439-9d69-4ad1-9cb2-cad1a02d2d22 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:52.102517 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca708439-9d69-4ad1-9cb2-cad1a02d2d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:52.121087 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-ca708439-9d69-4ad1-9cb2-cad1a02d2d22 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:52.122343 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-ca708439-9d69-4ad1-9cb2-cad1a02d2d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:52.141327 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034c259-798c-4cbd-a190-3d17561f160e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:52.142516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034c259-798c-4cbd-a190-3d17561f160e None admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:52.143013 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034c259-798c-4cbd-a190-3d17561f160e None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:52.143564 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034c259-798c-4cbd-a190-3d17561f160e None admin] PATH_INFO: `/v3/projects/e646d320aa5a4f78bfccaf941cf46478/users/8ec07ee1efba4e8089b94f24be338f86/roles/3298581080b940c1ab0a41e5bbc8953f` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:52.146259 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f034c259-798c-4cbd-a190-3d17561f160e None admin] RBAC: Authorizing `identity:create_grant(project_id=e646d320aa5a4f78bfccaf941cf46478, user_id=8ec07ee1efba4e8089b94f24be338f86, role_id=3298581080b940c1ab0a41e5bbc8953f)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:52.148064 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f034c259-798c-4cbd-a190-3d17561f160e None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:52.157523 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:52.159813 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:52.167236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 475/961] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 27 04:09:51 2022] PUT /identity/v3/projects/e646d320aa5a4f78bfccaf941cf46478/users/8ec07ee1efba4e8089b94f24be338f86/roles/3298581080b940c1ab0a41e5bbc8953f => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:09:52.227785 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 487/962] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 04:09:52.253233 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034c259-798c-4cbd-a190-3d17561f160e None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:52.370813 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034c259-798c-4cbd-a190-3d17561f160e None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:52.407936 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da58fd-c884-4ed0-b138-cc95d1e2ca1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:52.408487 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da58fd-c884-4ed0-b138-cc95d1e2ca1b None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:52.408773 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da58fd-c884-4ed0-b138-cc95d1e2ca1b None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:52.409048 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da58fd-c884-4ed0-b138-cc95d1e2ca1b None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:52.409741 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da58fd-c884-4ed0-b138-cc95d1e2ca1b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:52.411213 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da58fd-c884-4ed0-b138-cc95d1e2ca1b None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:52.413766 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: WARNING keystone.server.flask.common [None req-61da58fd-c884-4ed0-b138-cc95d1e2ca1b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 27 04:09:52.426787 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.sql.core [None req-61da58fd-c884-4ed0-b138-cc95d1e2ca1b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 27 04:09:52.426787 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 27 04:09:52.426787 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [parameters: {'user_id': '3c9c4e6e3ba04a6b8bbd358ef9860954', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 27 04:09:52.426787 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80240) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 27 04:09:52.444989 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-61da58fd-c884-4ed0-b138-cc95d1e2ca1b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 27 04:09:52.444989 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:52.444989 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 04:09:52.444989 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 04:09:52.444989 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 04:09:52.444989 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 04:09:52.444989 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 04:09:52.444989 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 04:09:52.444989 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 04:09:52.444989 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application conn.query(q) Jul 27 04:09:52.444989 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 04:09:52.444989 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 04:09:52.444989 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 04:09:52.444989 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application result.read() Jul 27 04:09:52.444989 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 04:09:52.445927 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 04:09:52.445927 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 04:09:52.445927 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 04:09:52.445927 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 04:09:52.445927 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 04:09:52.445927 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 04:09:52.445927 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 04:09:52.445927 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 27 04:09:52.445927 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:52.445927 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 27 04:09:52.445927 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:52.445927 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:52.445927 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 27 04:09:52.445927 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 27 04:09:52.445927 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 27 04:09:52.445927 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 27 04:09:52.445927 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 04:09:52.445927 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application next(self.gen) Jul 27 04:09:52.446922 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 27 04:09:52.446922 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application yield resource Jul 27 04:09:52.446922 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 04:09:52.446922 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application next(self.gen) Jul 27 04:09:52.446922 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 27 04:09:52.446922 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self.session.rollback() Jul 27 04:09:52.446922 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 27 04:09:52.446922 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self.force_reraise() Jul 27 04:09:52.446922 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 27 04:09:52.446922 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise self.value Jul 27 04:09:52.446922 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 27 04:09:52.446922 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 27 04:09:52.446922 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 27 04:09:52.446922 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application session.commit() Jul 27 04:09:52.446922 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 27 04:09:52.446922 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 27 04:09:52.446922 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 27 04:09:52.447825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._prepare_impl() Jul 27 04:09:52.447825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 27 04:09:52.447825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self.session.flush() Jul 27 04:09:52.447825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 27 04:09:52.447825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._flush(objects) Jul 27 04:09:52.447825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 27 04:09:52.447825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 27 04:09:52.447825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 27 04:09:52.447825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application compat.raise_( Jul 27 04:09:52.447825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 04:09:52.447825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception Jul 27 04:09:52.447825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 27 04:09:52.447825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application flush_context.execute() Jul 27 04:09:52.447825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 27 04:09:52.447825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rec.execute(self) Jul 27 04:09:52.447825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 27 04:09:52.447825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 27 04:09:52.448807 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 27 04:09:52.448807 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 27 04:09:52.448807 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 27 04:09:52.448807 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 27 04:09:52.448807 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 27 04:09:52.448807 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 27 04:09:52.448807 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 27 04:09:52.448807 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 27 04:09:52.448807 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 27 04:09:52.448807 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 27 04:09:52.448807 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 27 04:09:52.448807 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 27 04:09:52.448807 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 27 04:09:52.448807 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 27 04:09:52.448807 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 04:09:52.448807 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception Jul 27 04:09:52.449658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 04:09:52.449658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 04:09:52.449658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 04:09:52.449658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 04:09:52.449658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 04:09:52.449658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 04:09:52.449658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 04:09:52.449658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application conn.query(q) Jul 27 04:09:52.449658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 04:09:52.449658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 04:09:52.449658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 04:09:52.449658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application result.read() Jul 27 04:09:52.449658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 04:09:52.449658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 04:09:52.449658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 04:09:52.449658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 04:09:52.452117 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 04:09:52.452117 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 04:09:52.452117 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 04:09:52.452117 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 04:09:52.452117 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 27 04:09:52.452117 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 27 04:09:52.452117 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [parameters: {'user_id': '3c9c4e6e3ba04a6b8bbd358ef9860954', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 27 04:09:52.452117 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 27 04:09:52.452117 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:52.452117 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 04:09:52.452117 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:52.452117 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:52.452117 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:52.452117 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:52.452117 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:52.463488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:52.463488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:52.463488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:52.463488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:52.463488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:52.463488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:52.463488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:52.463488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 27 04:09:52.463488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 27 04:09:52.463488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:52.463488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:52.463488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:09:52.463488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:52.463488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:09:52.463488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:52.463488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 27 04:09:52.463488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 27 04:09:52.465513 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 27 04:09:52.465513 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 27 04:09:52.465513 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 27 04:09:52.465513 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 27 04:09:52.465513 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 27 04:09:52.465513 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:52.465513 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 476/963] 10.222.0.46 () {60 vars in 1131 bytes} [Wed Jul 27 04:09:52 2022] POST /identity/v3/users => generated 160 bytes in 194 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:52.476508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca708439-9d69-4ad1-9cb2-cad1a02d2d22 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:52.586581 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca708439-9d69-4ad1-9cb2-cad1a02d2d22 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:52.623586 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612693d6-c90b-488d-891a-a54d4ecc606e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:52.624246 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612693d6-c90b-488d-891a-a54d4ecc606e None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:52.624533 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612693d6-c90b-488d-891a-a54d4ecc606e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:52.624863 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612693d6-c90b-488d-891a-a54d4ecc606e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:52.632833 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612693d6-c90b-488d-891a-a54d4ecc606e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:52.634305 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612693d6-c90b-488d-891a-a54d4ecc606e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:52.646509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-612693d6-c90b-488d-891a-a54d4ecc606e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 04:09:52.646509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:52.646509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:52.646509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:52.646509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:52.646509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:52.646509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:52.646509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:52.646509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:52.646509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:52.646509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:52.646509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:52.646509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:09:52.646509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:09:52.646509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:09:52.646509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:09:52.647463 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:52.647463 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:52.647463 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:09:52.647463 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:52.647463 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:09:52.647463 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:52.647463 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:52.647463 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:52.647463 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:52.647463 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:52.647463 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:52.647463 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:52.647463 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:52.647463 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:52.647463 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:52.647463 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:52.647463 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:52.647463 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:52.648421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:52.648421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:52.648421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:09:52.648421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:09:52.648421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:09:52.648421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:09:52.648421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:09:52.648421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:09:52.648421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 04:09:52.648421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:52.648421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 488/964] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:09:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:09:52.655537 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da58fd-c884-4ed0-b138-cc95d1e2ca1b None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:52.724967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:52.768198 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da58fd-c884-4ed0-b138-cc95d1e2ca1b None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:52.803515 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa26b3-637a-4f49-8804-db6a362c28eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:52.804146 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa26b3-637a-4f49-8804-db6a362c28eb None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:52.804431 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa26b3-637a-4f49-8804-db6a362c28eb None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:52.804708 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa26b3-637a-4f49-8804-db6a362c28eb None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:52.805477 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa26b3-637a-4f49-8804-db6a362c28eb None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:52.805782 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa26b3-637a-4f49-8804-db6a362c28eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:52.807464 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa26b3-637a-4f49-8804-db6a362c28eb None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:52.846452 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 477/965] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 27 04:09:52 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:09:53.649413 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d44f8d-4ac2-47a1-8a66-858da84c1d88 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:53.649922 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d44f8d-4ac2-47a1-8a66-858da84c1d88 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:53.650256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d44f8d-4ac2-47a1-8a66-858da84c1d88 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:53.651390 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 489/966] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:53.656672 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19ae95-cfab-4edd-b6b3-09c787943654 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:53.657082 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19ae95-cfab-4edd-b6b3-09c787943654 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:53.657247 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19ae95-cfab-4edd-b6b3-09c787943654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:53.676438 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-aa19ae95-cfab-4edd-b6b3-09c787943654 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:53.677957 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-aa19ae95-cfab-4edd-b6b3-09c787943654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:53.747774 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 478/967] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:09:53.770840 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d44f8d-4ac2-47a1-8a66-858da84c1d88 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:53.797719 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:53.879367 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d44f8d-4ac2-47a1-8a66-858da84c1d88 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:53.914218 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54393b0-3e4d-4efc-b1e8-04819525523e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:53.914853 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54393b0-3e4d-4efc-b1e8-04819525523e None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:53.915141 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54393b0-3e4d-4efc-b1e8-04819525523e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:53.915418 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54393b0-3e4d-4efc-b1e8-04819525523e None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:53.916134 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54393b0-3e4d-4efc-b1e8-04819525523e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:53.917515 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54393b0-3e4d-4efc-b1e8-04819525523e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:53.929384 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da3100-2c25-4ea0-8067-9d07acb7f621 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:53.929695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da3100-2c25-4ea0-8067-9d07acb7f621 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:53.929937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da3100-2c25-4ea0-8067-9d07acb7f621 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:53.931309 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 479/968] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:53.933809 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 490/969] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Jul 27 04:09:53 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:09:53.939859 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85095ea9-8c40-488e-ab70-5015ee91712e None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:53.940146 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85095ea9-8c40-488e-ab70-5015ee91712e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:53.940422 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85095ea9-8c40-488e-ab70-5015ee91712e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:53.960871 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-85095ea9-8c40-488e-ab70-5015ee91712e None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:53.962347 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-85095ea9-8c40-488e-ab70-5015ee91712e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:54.018165 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:54.048735 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 480/970] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:53 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:09:54.071569 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54393b0-3e4d-4efc-b1e8-04819525523e None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:54.164202 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:54.184850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54393b0-3e4d-4efc-b1e8-04819525523e None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:54.221074 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d05e7-8a7a-4a5d-9536-182562cae67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:54.222108 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d05e7-8a7a-4a5d-9536-182562cae67c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:54.222624 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d05e7-8a7a-4a5d-9536-182562cae67c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:54.223124 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d05e7-8a7a-4a5d-9536-182562cae67c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:54.223990 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d05e7-8a7a-4a5d-9536-182562cae67c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:54.225657 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d05e7-8a7a-4a5d-9536-182562cae67c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:54.226716 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123dc7c-ed5b-4ec9-81be-80d5e50f2120 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:54.226997 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123dc7c-ed5b-4ec9-81be-80d5e50f2120 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:54.227270 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123dc7c-ed5b-4ec9-81be-80d5e50f2120 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:54.228607 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 481/971] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 04:09:54.233249 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055bb1f9-47bd-4a84-b580-2136116106d3 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:54.233529 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055bb1f9-47bd-4a84-b580-2136116106d3 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:54.233801 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055bb1f9-47bd-4a84-b580-2136116106d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:54.237536 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-7a3d05e7-8a7a-4a5d-9536-182562cae67c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 04:09:54.237536 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:54.237536 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 04:09:54.237536 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 04:09:54.237536 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 04:09:54.237536 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 04:09:54.237536 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 04:09:54.237536 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:54.237536 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 04:09:54.237536 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:54.237536 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 04:09:54.237536 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:54.237536 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:54.237536 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:54.237536 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:54.237536 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:54.237536 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:54.238678 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:54.238678 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:54.238678 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:54.238678 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:54.238678 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:54.238678 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:54.238678 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 04:09:54.238678 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 04:09:54.238678 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 04:09:54.238678 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 04:09:54.238678 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:54.238678 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:54.238678 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:54.238678 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:54.238678 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:54.238678 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:54.239551 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:54.239551 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:54.239551 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:54.239551 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:54.239551 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:54.239551 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:54.239551 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:54.239551 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:54.239551 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:54.239551 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:54.239551 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 04:09:54.239551 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:54.239551 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 04:09:54.239551 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:54.242682 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 491/972] 10.222.0.46 () {56 vars in 1126 bytes} [Wed Jul 27 04:09:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:54.245304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d05e7-8a7a-4a5d-9536-182562cae67c None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:54.253841 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-055bb1f9-47bd-4a84-b580-2136116106d3 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:54.255057 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-055bb1f9-47bd-4a84-b580-2136116106d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:54.329906 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 482/973] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:54 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 04:09:54.354053 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055bb1f9-47bd-4a84-b580-2136116106d3 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:54.361592 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d05e7-8a7a-4a5d-9536-182562cae67c None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:54.395937 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b0440-cd50-4575-9755-a64aff0740c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:54.396568 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b0440-cd50-4575-9755-a64aff0740c3 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:54.396856 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b0440-cd50-4575-9755-a64aff0740c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:54.397135 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b0440-cd50-4575-9755-a64aff0740c3 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:54.397818 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32b0440-cd50-4575-9755-a64aff0740c3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:54.398162 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32b0440-cd50-4575-9755-a64aff0740c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:54.399518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32b0440-cd50-4575-9755-a64aff0740c3 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:54.409605 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 492/974] 10.222.0.46 () {56 vars in 1122 bytes} [Wed Jul 27 04:09:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:54.411546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b0440-cd50-4575-9755-a64aff0740c3 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:54.428525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:54.478364 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055bb1f9-47bd-4a84-b580-2136116106d3 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:54.487865 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:54.520611 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72199c63-9817-4663-ae3f-8fd79a161145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:54.521278 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72199c63-9817-4663-ae3f-8fd79a161145 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:54.521546 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72199c63-9817-4663-ae3f-8fd79a161145 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:54.521810 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72199c63-9817-4663-ae3f-8fd79a161145 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:54.529229 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b0440-cd50-4575-9755-a64aff0740c3 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:54.532829 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72199c63-9817-4663-ae3f-8fd79a161145 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:09:54.537710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-72199c63-9817-4663-ae3f-8fd79a161145 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 04:09:54.537710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:54.537710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:54.537710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:54.537710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:54.537710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:54.537710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:54.537710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:54.537710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:54.537710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:54.537710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:54.537710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:54.537710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:09:54.537710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:09:54.537710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:09:54.537710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:09:54.538669 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:09:54.538669 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:09:54.538669 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:09:54.538669 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:09:54.538669 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:54.538669 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:54.538669 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:54.538669 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:54.538669 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:54.538669 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:54.538669 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:54.538669 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:54.538669 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:54.538669 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:54.538669 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:54.538669 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:54.539516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:54.539516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:54.539516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:54.539516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:54.539516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:09:54.539516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:09:54.539516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:09:54.539516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:09:54.539516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:09:54.539516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:09:54.539516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 04:09:54.539516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:54.542323 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 483/975] 10.222.0.46 () {56 vars in 1120 bytes} [Wed Jul 27 04:09:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:54.544746 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72199c63-9817-4663-ae3f-8fd79a161145 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:54.570639 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4152c-266d-4a8c-8898-de61fdaa6f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:54.571510 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4152c-266d-4a8c-8898-de61fdaa6f5b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:54.571797 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4152c-266d-4a8c-8898-de61fdaa6f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:54.572095 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4152c-266d-4a8c-8898-de61fdaa6f5b None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:54.573295 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4152c-266d-4a8c-8898-de61fdaa6f5b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:54.573721 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4152c-266d-4a8c-8898-de61fdaa6f5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:54.575759 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4152c-266d-4a8c-8898-de61fdaa6f5b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:54.586716 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 493/976] 10.222.0.46 () {56 vars in 1122 bytes} [Wed Jul 27 04:09:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:09:54.588194 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4152c-266d-4a8c-8898-de61fdaa6f5b None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:54.652011 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72199c63-9817-4663-ae3f-8fd79a161145 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:54.677587 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:54.691379 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facbb49a-3a8e-4e32-9bf6-bfa6d8c127af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:54.692081 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbb49a-3a8e-4e32-9bf6-bfa6d8c127af None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:54.692290 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbb49a-3a8e-4e32-9bf6-bfa6d8c127af None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:54.692563 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbb49a-3a8e-4e32-9bf6-bfa6d8c127af None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:54.693294 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facbb49a-3a8e-4e32-9bf6-bfa6d8c127af None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:54.693600 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facbb49a-3a8e-4e32-9bf6-bfa6d8c127af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:54.695005 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facbb49a-3a8e-4e32-9bf6-bfa6d8c127af None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:54.703044 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4152c-266d-4a8c-8898-de61fdaa6f5b None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:54.706154 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 484/977] 10.222.0.46 () {56 vars in 1115 bytes} [Wed Jul 27 04:09:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:09:54.709540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facbb49a-3a8e-4e32-9bf6-bfa6d8c127af None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:54.740765 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:54.746092 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbc966e-4b15-45bd-b89f-7cf0aacd6430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:54.746727 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc966e-4b15-45bd-b89f-7cf0aacd6430 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:54.747016 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc966e-4b15-45bd-b89f-7cf0aacd6430 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:54.747303 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc966e-4b15-45bd-b89f-7cf0aacd6430 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:54.748607 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbc966e-4b15-45bd-b89f-7cf0aacd6430 None admin] RBAC: Adding query filter params (interface=admin, service_id=b24e3edeb77c4c6cb924f9709225e28b, region_id=RegionOne) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:54.748607 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbc966e-4b15-45bd-b89f-7cf0aacd6430 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:54.749729 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbc966e-4b15-45bd-b89f-7cf0aacd6430 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:54.755467 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 494/978] 10.222.0.46 () {56 vars in 1249 bytes} [Wed Jul 27 04:09:54 2022] GET /identity/v3/endpoints?service_id=b24e3edeb77c4c6cb924f9709225e28b&interface=admin®ion_id=RegionOne => generated 193 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:09:54.760724 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbc966e-4b15-45bd-b89f-7cf0aacd6430 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:54.823691 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facbb49a-3a8e-4e32-9bf6-bfa6d8c127af None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:54.859291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec4adb-8bac-4c9e-be23-09b248dbd586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:54.859835 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec4adb-8bac-4c9e-be23-09b248dbd586 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:54.860112 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec4adb-8bac-4c9e-be23-09b248dbd586 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:54.860389 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec4adb-8bac-4c9e-be23-09b248dbd586 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:54.867015 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbc966e-4b15-45bd-b89f-7cf0aacd6430 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:54.871588 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec4adb-8bac-4c9e-be23-09b248dbd586 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:54.873094 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec4adb-8bac-4c9e-be23-09b248dbd586 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:54.888882 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-3eec4adb-8bac-4c9e-be23-09b248dbd586 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:54.888882 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:54.888882 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:54.888882 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:54.888882 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:54.888882 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:54.888882 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:54.888882 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:54.888882 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:54.888882 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:54.888882 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:54.888882 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:54.888882 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:54.888882 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:54.888882 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:54.889818 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:54.889818 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:54.889818 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:54.889818 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:54.889818 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:54.889818 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:54.889818 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:54.889818 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:54.889818 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:54.889818 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:54.889818 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:54.889818 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:54.889818 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:54.889818 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:54.889818 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:54.889818 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:54.889818 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:54.891695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:54.891695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:54.891695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:54.891695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:54.891695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 485/979] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:54.895456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec4adb-8bac-4c9e-be23-09b248dbd586 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:54.905242 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d487f-831a-44c7-8845-995d2e0d68dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:54.905862 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d487f-831a-44c7-8845-995d2e0d68dc None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:54.906798 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d487f-831a-44c7-8845-995d2e0d68dc None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:54.907297 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d487f-831a-44c7-8845-995d2e0d68dc None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:54.907866 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d487f-831a-44c7-8845-995d2e0d68dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:54.909224 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d487f-831a-44c7-8845-995d2e0d68dc None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:54.915788 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 495/980] 10.222.0.46 () {56 vars in 1087 bytes} [Wed Jul 27 04:09:54 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 04:09:54.986144 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:55.008558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec4adb-8bac-4c9e-be23-09b248dbd586 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:55.045141 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73053961-5e10-4b65-8ec3-b1317f46823d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:55.045777 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73053961-5e10-4b65-8ec3-b1317f46823d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:55.046083 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73053961-5e10-4b65-8ec3-b1317f46823d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:55.046358 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73053961-5e10-4b65-8ec3-b1317f46823d None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:55.047759 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73053961-5e10-4b65-8ec3-b1317f46823d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:55.048063 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73053961-5e10-4b65-8ec3-b1317f46823d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:55.049474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73053961-5e10-4b65-8ec3-b1317f46823d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:55.062897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 486/981] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:55.068908 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d487f-831a-44c7-8845-995d2e0d68dc None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:55.074402 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:55.176556 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d487f-831a-44c7-8845-995d2e0d68dc None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:55.210999 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8295d7f-261d-49c2-a5d2-3a2ca46bf05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:55.211765 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8295d7f-261d-49c2-a5d2-3a2ca46bf05d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:55.212054 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8295d7f-261d-49c2-a5d2-3a2ca46bf05d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:55.212336 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8295d7f-261d-49c2-a5d2-3a2ca46bf05d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:55.220216 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8295d7f-261d-49c2-a5d2-3a2ca46bf05d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:55.221653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8295d7f-261d-49c2-a5d2-3a2ca46bf05d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:55.232951 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-f8295d7f-261d-49c2-a5d2-3a2ca46bf05d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 04:09:55.232951 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:55.232951 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:55.232951 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:55.232951 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:55.232951 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:55.232951 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:55.232951 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:55.232951 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:55.232951 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:55.232951 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:55.232951 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:55.232951 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:09:55.232951 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:09:55.232951 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:09:55.232951 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:09:55.233882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:55.233882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:55.233882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:09:55.233882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:55.233882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:09:55.233882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:09:55.233882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:55.233882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:55.233882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:55.233882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:55.233882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:55.233882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:55.233882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:55.233882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:55.233882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:55.233882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:55.233882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:55.233882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:55.236196 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:55.236196 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:55.236196 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:09:55.236196 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:09:55.236196 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:09:55.236196 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:09:55.236196 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:09:55.236196 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:09:55.236196 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 04:09:55.236196 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:55.236196 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 496/982] 10.222.0.46 () {56 vars in 1117 bytes} [Wed Jul 27 04:09:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:55.242705 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73053961-5e10-4b65-8ec3-b1317f46823d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:55.345005 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73053961-5e10-4b65-8ec3-b1317f46823d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:55.378812 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b07a9-73ea-4af6-9c61-dbf0c69cc6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:55.379434 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b07a9-73ea-4af6-9c61-dbf0c69cc6cb None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:55.379713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b07a9-73ea-4af6-9c61-dbf0c69cc6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:55.379984 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b07a9-73ea-4af6-9c61-dbf0c69cc6cb None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:55.380777 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b07a9-73ea-4af6-9c61-dbf0c69cc6cb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:55.381079 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b07a9-73ea-4af6-9c61-dbf0c69cc6cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:55.382538 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b07a9-73ea-4af6-9c61-dbf0c69cc6cb None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:55.392868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:55.407726 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 487/983] 10.222.0.46 () {56 vars in 1149 bytes} [Wed Jul 27 04:09:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:55.408989 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8295d7f-261d-49c2-a5d2-3a2ca46bf05d None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:55.482017 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:55.523812 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8295d7f-261d-49c2-a5d2-3a2ca46bf05d None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:55.558515 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01dc745-b9d0-4d09-b77b-e0bd1fe7e92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:55.559149 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01dc745-b9d0-4d09-b77b-e0bd1fe7e92e None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:55.559434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01dc745-b9d0-4d09-b77b-e0bd1fe7e92e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:55.559716 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01dc745-b9d0-4d09-b77b-e0bd1fe7e92e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:55.629052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01dc745-b9d0-4d09-b77b-e0bd1fe7e92e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:55.630617 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01dc745-b9d0-4d09-b77b-e0bd1fe7e92e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:55.691961 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:55.737161 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 497/984] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:09:55 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 04:09:55.738717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b07a9-73ea-4af6-9c61-dbf0c69cc6cb None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:55.839429 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b07a9-73ea-4af6-9c61-dbf0c69cc6cb None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:55.873664 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2a2e6-2bca-43d7-85e9-82d1d20ca614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:55.874600 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2a2e6-2bca-43d7-85e9-82d1d20ca614 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:55.875150 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2a2e6-2bca-43d7-85e9-82d1d20ca614 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:55.875629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2a2e6-2bca-43d7-85e9-82d1d20ca614 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:55.886069 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a2a2e6-2bca-43d7-85e9-82d1d20ca614 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:55.887693 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a2a2e6-2bca-43d7-85e9-82d1d20ca614 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:55.903291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-69a2a2e6-2bca-43d7-85e9-82d1d20ca614 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:55.903291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:55.903291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:55.903291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:55.903291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:55.903291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:55.903291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:55.903291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:55.903291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:55.903291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:55.903291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:55.903291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:55.903291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:09:55.903291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:09:55.903291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:09:55.904290 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:09:55.904290 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:55.904290 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:55.904290 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:55.904290 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:55.904290 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:55.904290 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:55.904290 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:55.904290 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:55.904290 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:55.904290 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:55.904290 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:55.904290 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:55.904290 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:55.904290 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:55.904290 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:55.904290 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:55.905198 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:09:55.905198 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:09:55.905198 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:09:55.905198 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:55.905472 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 488/985] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:55.911548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01dc745-b9d0-4d09-b77b-e0bd1fe7e92e None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:55.977096 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d6c5a-eaec-4611-9e9a-4769f845671d None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:55.977442 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d6c5a-eaec-4611-9e9a-4769f845671d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:55.977653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d6c5a-eaec-4611-9e9a-4769f845671d None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:55.979174 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 489/986] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 04:09:55.983879 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46acfc44-d4b9-4509-9ab1-ec6c613de355 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:55.984191 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46acfc44-d4b9-4509-9ab1-ec6c613de355 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:55.984556 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46acfc44-d4b9-4509-9ab1-ec6c613de355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:56.002726 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-46acfc44-d4b9-4509-9ab1-ec6c613de355 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:56.003844 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-46acfc44-d4b9-4509-9ab1-ec6c613de355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:56.016359 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01dc745-b9d0-4d09-b77b-e0bd1fe7e92e None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:56.054632 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8097bd8a-98f6-4fb5-84ef-4d1742af61e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:56.055257 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097bd8a-98f6-4fb5-84ef-4d1742af61e6 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:56.055537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097bd8a-98f6-4fb5-84ef-4d1742af61e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:56.055812 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097bd8a-98f6-4fb5-84ef-4d1742af61e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:56.056514 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8097bd8a-98f6-4fb5-84ef-4d1742af61e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:56.056814 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8097bd8a-98f6-4fb5-84ef-4d1742af61e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:56.056917 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:56.058296 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8097bd8a-98f6-4fb5-84ef-4d1742af61e6 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:56.071009 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 498/987] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:09:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:56.076377 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8097bd8a-98f6-4fb5-84ef-4d1742af61e6 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:56.085588 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 490/988] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:55 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 04:09:56.103119 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:56.110303 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46acfc44-d4b9-4509-9ab1-ec6c613de355 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:56.184387 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8097bd8a-98f6-4fb5-84ef-4d1742af61e6 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:56.216790 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46acfc44-d4b9-4509-9ab1-ec6c613de355 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:56.219124 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d489e-1ec0-4d47-9970-4e5f98031749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:56.219772 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d489e-1ec0-4d47-9970-4e5f98031749 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:56.220051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d489e-1ec0-4d47-9970-4e5f98031749 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:56.220319 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d489e-1ec0-4d47-9970-4e5f98031749 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:56.226989 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d489e-1ec0-4d47-9970-4e5f98031749 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:56.228416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d489e-1ec0-4d47-9970-4e5f98031749 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:56.240197 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-4c0d489e-1ec0-4d47-9970-4e5f98031749 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:56.240197 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:56.240197 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:56.240197 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:56.240197 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:56.240197 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:56.240197 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:56.240197 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:56.240197 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:56.240197 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:56.240197 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:56.240197 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:56.240197 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:09:56.240197 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:09:56.240197 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:09:56.241302 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:09:56.241302 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:56.241302 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:56.241302 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:56.241302 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:56.241302 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:56.241302 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:56.241302 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:56.241302 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:56.241302 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:56.241302 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:56.241302 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:56.241302 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:56.241302 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:56.241302 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:56.241302 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:56.241302 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:56.247079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:09:56.247079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:09:56.247079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:09:56.247079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:09:56.247079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:09:56.247079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:09:56.247079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:09:56.247079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:56.247079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 499/989] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:56.247079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d489e-1ec0-4d47-9970-4e5f98031749 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:56.261184 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e1a7b-ffca-4d28-917b-974a2c2ab87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:56.261763 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e1a7b-ffca-4d28-917b-974a2c2ab87e None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:56.262064 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e1a7b-ffca-4d28-917b-974a2c2ab87e None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:56.262357 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e1a7b-ffca-4d28-917b-974a2c2ab87e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:56.262975 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610e1a7b-ffca-4d28-917b-974a2c2ab87e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:56.264329 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610e1a7b-ffca-4d28-917b-974a2c2ab87e None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:56.273947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-610e1a7b-ffca-4d28-917b-974a2c2ab87e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 04:09:56.273947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:56.273947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 04:09:56.273947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 04:09:56.273947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 04:09:56.273947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 04:09:56.273947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 04:09:56.273947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:56.273947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 04:09:56.273947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:56.273947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 04:09:56.273947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:56.273947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:56.273947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:56.273947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:56.273947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:56.273947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:56.275007 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:56.275007 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:56.275007 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:56.275007 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:56.275007 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:56.275007 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:56.275007 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 04:09:56.275007 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 04:09:56.275007 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 04:09:56.275007 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 04:09:56.275007 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:56.275007 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:56.275007 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:56.275007 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:56.275007 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:56.275007 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:56.275889 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:56.275889 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:56.275889 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:56.275889 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:56.275889 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:56.275889 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:56.275889 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:56.275889 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:56.275889 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:56.275889 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:56.275889 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 04:09:56.275889 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:56.275889 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 04:09:56.275889 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:56.275889 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 491/990] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:56.280900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e1a7b-ffca-4d28-917b-974a2c2ab87e None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:56.328800 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:56.340252 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:56.357713 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d489e-1ec0-4d47-9970-4e5f98031749 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:56.387066 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e1a7b-ffca-4d28-917b-974a2c2ab87e None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:56.392830 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b75e6-0144-4e36-a7e8-effde7f2a31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:56.393451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b75e6-0144-4e36-a7e8-effde7f2a31d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:56.393731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b75e6-0144-4e36-a7e8-effde7f2a31d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:56.394026 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b75e6-0144-4e36-a7e8-effde7f2a31d None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:56.394788 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b75e6-0144-4e36-a7e8-effde7f2a31d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:56.395094 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b75e6-0144-4e36-a7e8-effde7f2a31d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:56.396512 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b75e6-0144-4e36-a7e8-effde7f2a31d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:56.413880 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 500/991] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:09:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:09:56.415049 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b75e6-0144-4e36-a7e8-effde7f2a31d None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:56.428134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d683b3c-27c7-4a89-99ce-373f003ba8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:56.428836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d683b3c-27c7-4a89-99ce-373f003ba8a5 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:56.429187 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d683b3c-27c7-4a89-99ce-373f003ba8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:56.429616 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d683b3c-27c7-4a89-99ce-373f003ba8a5 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:56.430515 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d683b3c-27c7-4a89-99ce-373f003ba8a5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:56.430860 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d683b3c-27c7-4a89-99ce-373f003ba8a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:56.432278 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d683b3c-27c7-4a89-99ce-373f003ba8a5 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:56.438021 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 492/992] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Jul 27 04:09:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:09:56.443923 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d683b3c-27c7-4a89-99ce-373f003ba8a5 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:56.536053 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b75e6-0144-4e36-a7e8-effde7f2a31d None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:56.560694 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d683b3c-27c7-4a89-99ce-373f003ba8a5 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:56.570648 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:56.573451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2dfc80-b74b-4c4d-a435-8e3e53c7e504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:56.574175 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2dfc80-b74b-4c4d-a435-8e3e53c7e504 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:56.574466 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2dfc80-b74b-4c4d-a435-8e3e53c7e504 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:56.574747 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2dfc80-b74b-4c4d-a435-8e3e53c7e504 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:56.575559 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2dfc80-b74b-4c4d-a435-8e3e53c7e504 None admin] RBAC: Adding query filter params (role.id=3298581080b940c1ab0a41e5bbc8953f, scope.project.id=e646d320aa5a4f78bfccaf941cf46478, user.id=8ec07ee1efba4e8089b94f24be338f86) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:56.575876 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2dfc80-b74b-4c4d-a435-8e3e53c7e504 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:56.577313 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2dfc80-b74b-4c4d-a435-8e3e53c7e504 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:56.584319 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 501/993] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:09:56 2022] GET /identity/v3/role_assignments?user.id=8ec07ee1efba4e8089b94f24be338f86&scope.project.id=e646d320aa5a4f78bfccaf941cf46478&role.id=3298581080b940c1ab0a41e5bbc8953f => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:56.599393 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758de869-b6d3-46d6-80c6-56f9c5078152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:56.600105 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758de869-b6d3-46d6-80c6-56f9c5078152 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:56.600389 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758de869-b6d3-46d6-80c6-56f9c5078152 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:56.600668 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758de869-b6d3-46d6-80c6-56f9c5078152 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:56.601383 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758de869-b6d3-46d6-80c6-56f9c5078152 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:56.601685 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758de869-b6d3-46d6-80c6-56f9c5078152 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:56.603142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758de869-b6d3-46d6-80c6-56f9c5078152 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:56.614624 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 493/994] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Jul 27 04:09:56 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:09:56.619133 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2dfc80-b74b-4c4d-a435-8e3e53c7e504 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:56.632644 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:56.735423 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2dfc80-b74b-4c4d-a435-8e3e53c7e504 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:56.772764 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c67092-246b-487f-a19f-e1ffd7e83854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:56.773402 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c67092-246b-487f-a19f-e1ffd7e83854 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:56.773692 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c67092-246b-487f-a19f-e1ffd7e83854 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:56.774050 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c67092-246b-487f-a19f-e1ffd7e83854 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:56.774784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c67092-246b-487f-a19f-e1ffd7e83854 None admin] RBAC: Adding query filter params (interface=public, service_id=defd32904d4441dc90555cade438377f, region_id=RegionOne) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:56.775100 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c67092-246b-487f-a19f-e1ffd7e83854 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:56.776464 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c67092-246b-487f-a19f-e1ffd7e83854 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:56.782311 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 502/995] 10.222.0.46 () {56 vars in 1251 bytes} [Wed Jul 27 04:09:56 2022] GET /identity/v3/endpoints?service_id=defd32904d4441dc90555cade438377f&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:56.788041 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758de869-b6d3-46d6-80c6-56f9c5078152 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:56.895675 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758de869-b6d3-46d6-80c6-56f9c5078152 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:56.929221 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c38014-d4a8-4e08-acbd-38949c1cee46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:56.929847 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c38014-d4a8-4e08-acbd-38949c1cee46 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:56.930152 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c38014-d4a8-4e08-acbd-38949c1cee46 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:56.930426 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c38014-d4a8-4e08-acbd-38949c1cee46 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:56.931090 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c38014-d4a8-4e08-acbd-38949c1cee46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:56.932450 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c38014-d4a8-4e08-acbd-38949c1cee46 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:56.939740 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 494/996] 10.222.0.46 () {56 vars in 1087 bytes} [Wed Jul 27 04:09:56 2022] GET /identity/v3/services => generated 1727 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:09:56.951915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b1fab-8024-4c24-b2ae-ff37523cb5e4 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:56.952206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b1fab-8024-4c24-b2ae-ff37523cb5e4 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:56.952490 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b1fab-8024-4c24-b2ae-ff37523cb5e4 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:56.954719 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 503/997] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:56.960636 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed599d06-18c8-4e56-a8c5-e8d4558916db None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:56.960968 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed599d06-18c8-4e56-a8c5-e8d4558916db None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:56.961185 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed599d06-18c8-4e56-a8c5-e8d4558916db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:56.980126 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-ed599d06-18c8-4e56-a8c5-e8d4558916db None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:56.981470 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ed599d06-18c8-4e56-a8c5-e8d4558916db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:57.014717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:57.062119 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 495/998] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:56 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:09:57.083631 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b1fab-8024-4c24-b2ae-ff37523cb5e4 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:57.159137 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:57.192656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b1fab-8024-4c24-b2ae-ff37523cb5e4 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:57.226032 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df46e91-50bb-4077-b171-8f7ec31514bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:57.226677 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df46e91-50bb-4077-b171-8f7ec31514bd None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:57.226957 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df46e91-50bb-4077-b171-8f7ec31514bd None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:57.227231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df46e91-50bb-4077-b171-8f7ec31514bd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:57.227861 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df46e91-50bb-4077-b171-8f7ec31514bd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:57.229209 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df46e91-50bb-4077-b171-8f7ec31514bd None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:57.238562 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-4df46e91-50bb-4077-b171-8f7ec31514bd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 04:09:57.238562 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:57.238562 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 04:09:57.238562 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 04:09:57.238562 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 04:09:57.238562 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 04:09:57.238562 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 04:09:57.238562 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:57.238562 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 04:09:57.238562 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:57.238562 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 04:09:57.238562 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:57.238562 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:57.238562 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:57.238562 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:57.238562 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:57.238562 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:57.239555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:57.239555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:57.239555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:57.239555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:57.239555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:57.239555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:57.239555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 04:09:57.239555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 04:09:57.239555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 04:09:57.239555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 04:09:57.239555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:57.239555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:57.239555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:57.239555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:57.239555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:57.239555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:57.240409 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:57.240409 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:57.240409 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:57.240409 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:57.240409 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:57.240409 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:57.240409 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:57.240409 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:57.240409 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:57.240409 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:57.240409 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 04:09:57.240409 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:57.240409 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 04:09:57.240409 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:57.240409 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 504/999] 10.222.0.46 () {56 vars in 1126 bytes} [Wed Jul 27 04:09:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:57.247220 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed599d06-18c8-4e56-a8c5-e8d4558916db None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:57.352556 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed599d06-18c8-4e56-a8c5-e8d4558916db None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:57.386197 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c259750-88aa-4d2a-b613-b4405f60e279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:57.386822 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c259750-88aa-4d2a-b613-b4405f60e279 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:57.387101 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c259750-88aa-4d2a-b613-b4405f60e279 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:57.387369 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c259750-88aa-4d2a-b613-b4405f60e279 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:57.388049 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c259750-88aa-4d2a-b613-b4405f60e279 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:57.388404 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c259750-88aa-4d2a-b613-b4405f60e279 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:57.389819 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c259750-88aa-4d2a-b613-b4405f60e279 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:57.399771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 496/1000] 10.222.0.46 () {56 vars in 1122 bytes} [Wed Jul 27 04:09:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:57.401047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df46e91-50bb-4077-b171-8f7ec31514bd None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:57.510823 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df46e91-50bb-4077-b171-8f7ec31514bd None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:57.544694 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a11e14f-bf9e-49e5-95d6-fb94e2780b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:57.545316 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11e14f-bf9e-49e5-95d6-fb94e2780b85 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:57.545601 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11e14f-bf9e-49e5-95d6-fb94e2780b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:57.545876 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11e14f-bf9e-49e5-95d6-fb94e2780b85 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:57.547142 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a11e14f-bf9e-49e5-95d6-fb94e2780b85 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:57.547464 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a11e14f-bf9e-49e5-95d6-fb94e2780b85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:57.548824 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a11e14f-bf9e-49e5-95d6-fb94e2780b85 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:57.551159 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:57.554532 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 505/1001] 10.222.0.46 () {56 vars in 1122 bytes} [Wed Jul 27 04:09:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:09:57.560512 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c259750-88aa-4d2a-b613-b4405f60e279 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:57.592245 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:57.665291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c259750-88aa-4d2a-b613-b4405f60e279 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:57.700073 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63b305-cede-4dbf-ba69-115a7ea5f6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:57.700706 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63b305-cede-4dbf-ba69-115a7ea5f6bc None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:57.700991 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63b305-cede-4dbf-ba69-115a7ea5f6bc None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:57.701267 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63b305-cede-4dbf-ba69-115a7ea5f6bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:57.702022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c63b305-cede-4dbf-ba69-115a7ea5f6bc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:57.703450 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c63b305-cede-4dbf-ba69-115a7ea5f6bc None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:57.721587 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 497/1002] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:09:57 2022] POST /identity/v3/endpoints => generated 325 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:09:58.675868 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5365cab-0244-4c40-9451-9798678f2276 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:58.676381 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5365cab-0244-4c40-9451-9798678f2276 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:58.676459 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5365cab-0244-4c40-9451-9798678f2276 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:58.677948 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 506/1003] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:58.683025 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349df818-271d-4615-9c6d-65f8d27ff773 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:58.683437 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349df818-271d-4615-9c6d-65f8d27ff773 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:58.683721 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349df818-271d-4615-9c6d-65f8d27ff773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:58.704641 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-349df818-271d-4615-9c6d-65f8d27ff773 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:58.706277 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-349df818-271d-4615-9c6d-65f8d27ff773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:58.782063 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 498/1004] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:58 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:09:58.804338 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5365cab-0244-4c40-9451-9798678f2276 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:58.908620 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5365cab-0244-4c40-9451-9798678f2276 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:58.943860 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ceddf7-d47d-4dce-977f-7f9144e92010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:58.944531 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ceddf7-d47d-4dce-977f-7f9144e92010 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:58.944807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ceddf7-d47d-4dce-977f-7f9144e92010 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:58.945087 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ceddf7-d47d-4dce-977f-7f9144e92010 None admin] PATH_INFO: `/v3/services/image` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:58.945731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ceddf7-d47d-4dce-977f-7f9144e92010 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:58.947126 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ceddf7-d47d-4dce-977f-7f9144e92010 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:58.949432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059dd919-3cd1-4a02-a98f-f943281a5f6e None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:58.949720 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059dd919-3cd1-4a02-a98f-f943281a5f6e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:58.950021 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059dd919-3cd1-4a02-a98f-f943281a5f6e None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:58.951404 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 499/1005] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 04:09:58.956869 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9887701-6dbb-4a71-bb4a-19c584c580ad None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:58.957096 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9887701-6dbb-4a71-bb4a-19c584c580ad None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:58.957377 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9887701-6dbb-4a71-bb4a-19c584c580ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:58.960428 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-20ceddf7-d47d-4dce-977f-7f9144e92010 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 04:09:58.960428 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:58.960428 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 04:09:58.960428 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 04:09:58.960428 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 04:09:58.960428 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 04:09:58.960428 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 04:09:58.960428 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:58.960428 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 04:09:58.960428 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:58.960428 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 04:09:58.960428 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:58.960428 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:58.960428 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:58.960428 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:58.960428 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:58.960428 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:58.961470 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:58.961470 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:58.961470 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:58.961470 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:58.961470 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:58.961470 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:58.961470 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 04:09:58.961470 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 04:09:58.961470 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 04:09:58.961470 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 04:09:58.961470 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:58.961470 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:58.961470 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:58.961470 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:58.961470 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:58.961470 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:58.962524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:58.962524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:58.962524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:58.962524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:58.962524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:58.962524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:58.962524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:58.962524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:58.962524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:58.962524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:58.962524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 04:09:58.962524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:58.962524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 04:09:58.962524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:09:58.964645 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 507/1006] 10.222.0.46 () {56 vars in 1105 bytes} [Wed Jul 27 04:09:58 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:09:58.971034 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ceddf7-d47d-4dce-977f-7f9144e92010 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:58.976593 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-c9887701-6dbb-4a71-bb4a-19c584c580ad None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:58.977918 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c9887701-6dbb-4a71-bb4a-19c584c580ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:58.989695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:59.035344 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:59.047807 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 500/1007] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:58 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 04:09:59.069658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9887701-6dbb-4a71-bb4a-19c584c580ad None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:59.076445 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ceddf7-d47d-4dce-977f-7f9144e92010 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:59.112382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ee8dc-0a20-4986-beb7-121c18baf9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:59.113020 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ee8dc-0a20-4986-beb7-121c18baf9f5 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:59.113314 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ee8dc-0a20-4986-beb7-121c18baf9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:59.113584 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ee8dc-0a20-4986-beb7-121c18baf9f5 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:59.114305 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ee8dc-0a20-4986-beb7-121c18baf9f5 None admin] RBAC: Adding query filter params (name=image) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:59.114615 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ee8dc-0a20-4986-beb7-121c18baf9f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:59.115979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ee8dc-0a20-4986-beb7-121c18baf9f5 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:59.120756 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 508/1008] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:09:59.126566 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ee8dc-0a20-4986-beb7-121c18baf9f5 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:59.171360 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9887701-6dbb-4a71-bb4a-19c584c580ad None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:59.209880 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d41d5-88bd-4f31-ba9d-efb4f6343ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:59.210859 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d41d5-88bd-4f31-ba9d-efb4f6343ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:59.211302 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d41d5-88bd-4f31-ba9d-efb4f6343ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:59.211788 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d41d5-88bd-4f31-ba9d-efb4f6343ad8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:59.212428 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33d41d5-88bd-4f31-ba9d-efb4f6343ad8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:59.213790 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33d41d5-88bd-4f31-ba9d-efb4f6343ad8 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:59.223433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-e33d41d5-88bd-4f31-ba9d-efb4f6343ad8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 04:09:59.223433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:59.223433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 04:09:59.223433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 04:09:59.223433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 04:09:59.223433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 04:09:59.223433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 04:09:59.223433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:59.223433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 04:09:59.223433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:59.223433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 04:09:59.223433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:59.223433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:09:59.223433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:09:59.223433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:09:59.223433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:09:59.223433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:09:59.224444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:09:59.224444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:09:59.224444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:09:59.224444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:09:59.224444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:09:59.224444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:09:59.224444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 04:09:59.224444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 04:09:59.224444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 04:09:59.224444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 04:09:59.224444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:09:59.224444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:09:59.224444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:09:59.224444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:09:59.224444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:09:59.224444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:09:59.225311 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:09:59.225311 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:09:59.225311 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:09:59.225311 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:09:59.225311 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:09:59.225311 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:09:59.225311 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:09:59.225311 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:09:59.225311 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:09:59.225311 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:09:59.225311 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 04:09:59.225311 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:09:59.225311 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 04:09:59.225311 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:09:59.225311 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 501/1009] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:09:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:09:59.230299 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d41d5-88bd-4f31-ba9d-efb4f6343ad8 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:59.232708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ee8dc-0a20-4986-beb7-121c18baf9f5 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:59.243571 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:59.261399 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:59.270373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741abfd9-1a45-4598-960b-653c5a04537e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:59.271000 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741abfd9-1a45-4598-960b-653c5a04537e None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:59.271278 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741abfd9-1a45-4598-960b-653c5a04537e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:59.271549 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741abfd9-1a45-4598-960b-653c5a04537e None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:59.272227 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741abfd9-1a45-4598-960b-653c5a04537e None admin] RBAC: Adding query filter params (type=image) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:59.272528 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741abfd9-1a45-4598-960b-653c5a04537e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:59.273927 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741abfd9-1a45-4598-960b-653c5a04537e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:59.279326 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 509/1010] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:59 2022] GET /identity/v3/services?type=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:09:59.342325 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d41d5-88bd-4f31-ba9d-efb4f6343ad8 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:59.379529 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60271fc-57c4-4498-9fb0-e1bb5db07fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:59.380169 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60271fc-57c4-4498-9fb0-e1bb5db07fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:59.380446 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60271fc-57c4-4498-9fb0-e1bb5db07fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:59.380716 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60271fc-57c4-4498-9fb0-e1bb5db07fc0 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:59.381408 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60271fc-57c4-4498-9fb0-e1bb5db07fc0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:59.381708 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60271fc-57c4-4498-9fb0-e1bb5db07fc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:59.383142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60271fc-57c4-4498-9fb0-e1bb5db07fc0 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:59.388810 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 502/1011] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Jul 27 04:09:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:09:59.394558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741abfd9-1a45-4598-960b-653c5a04537e None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:59.510915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741abfd9-1a45-4598-960b-653c5a04537e None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:59.543926 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4084fb8-7836-434a-a7ef-3c096d93a6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:59.544598 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4084fb8-7836-434a-a7ef-3c096d93a6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:59.544885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4084fb8-7836-434a-a7ef-3c096d93a6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:59.545158 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4084fb8-7836-434a-a7ef-3c096d93a6c9 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:59.545833 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4084fb8-7836-434a-a7ef-3c096d93a6c9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:09:59.546173 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4084fb8-7836-434a-a7ef-3c096d93a6c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:59.548072 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4084fb8-7836-434a-a7ef-3c096d93a6c9 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:59.553007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 510/1012] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Jul 27 04:09:59 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:09:59.558788 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60271fc-57c4-4498-9fb0-e1bb5db07fc0 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:59.638180 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:59.664323 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60271fc-57c4-4498-9fb0-e1bb5db07fc0 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:09:59.698607 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98328bbd-d6f3-432c-b03f-e3bc97d2ec77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:09:59.699278 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98328bbd-d6f3-432c-b03f-e3bc97d2ec77 None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:59.699523 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98328bbd-d6f3-432c-b03f-e3bc97d2ec77 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:59.699759 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98328bbd-d6f3-432c-b03f-e3bc97d2ec77 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:59.700484 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98328bbd-d6f3-432c-b03f-e3bc97d2ec77 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:09:59.701849 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98328bbd-d6f3-432c-b03f-e3bc97d2ec77 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:09:59.725104 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 503/1013] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:09:59 2022] POST /identity/v3/endpoints => generated 334 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:09:59.745738 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba241ff9-5cfe-4b17-b420-c65e17645932 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:59.746296 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba241ff9-5cfe-4b17-b420-c65e17645932 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:59.746374 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba241ff9-5cfe-4b17-b420-c65e17645932 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:59.747928 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 511/1014] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:09:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:09:59.754173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ef406-d63d-48fb-8dd1-ff86d1c18f66 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:09:59.754532 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ef406-d63d-48fb-8dd1-ff86d1c18f66 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:09:59.754732 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ef406-d63d-48fb-8dd1-ff86d1c18f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:09:59.774580 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-3d8ef406-d63d-48fb-8dd1-ff86d1c18f66 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:09:59.775707 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3d8ef406-d63d-48fb-8dd1-ff86d1c18f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:09:59.858123 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 504/1015] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:09:59 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:09:59.879846 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba241ff9-5cfe-4b17-b420-c65e17645932 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:09:59.942784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:09:59.991239 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba241ff9-5cfe-4b17-b420-c65e17645932 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:00.026056 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9877495b-bca5-4889-a1c8-f36024bb5950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:00.026690 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877495b-bca5-4889-a1c8-f36024bb5950 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:00.026976 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877495b-bca5-4889-a1c8-f36024bb5950 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:00.027250 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877495b-bca5-4889-a1c8-f36024bb5950 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:00.028409 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9877495b-bca5-4889-a1c8-f36024bb5950 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:00.029833 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9877495b-bca5-4889-a1c8-f36024bb5950 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:00.032313 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 512/1016] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:09:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:00.037845 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ef406-d63d-48fb-8dd1-ff86d1c18f66 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:00.096114 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:00.142683 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ef406-d63d-48fb-8dd1-ff86d1c18f66 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:00.177560 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f63353-e75b-4f98-b16e-f15fbd0e744f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:00.178204 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f63353-e75b-4f98-b16e-f15fbd0e744f None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:00.178508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f63353-e75b-4f98-b16e-f15fbd0e744f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:00.178783 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f63353-e75b-4f98-b16e-f15fbd0e744f None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:00.179478 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f63353-e75b-4f98-b16e-f15fbd0e744f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:00.180954 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f63353-e75b-4f98-b16e-f15fbd0e744f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:00.208527 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 505/1017] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Jul 27 04:10:00 2022] POST /identity/v3/projects => generated 311 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:10:01.306781 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1af0da-1126-4221-8198-5f2cefcf4a21 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:01.307287 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1af0da-1126-4221-8198-5f2cefcf4a21 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:01.307366 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1af0da-1126-4221-8198-5f2cefcf4a21 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:01.308722 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 513/1018] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:10:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:10:01.314298 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144dc2a-18ce-4276-8cc9-c02932de428b None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:01.314748 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144dc2a-18ce-4276-8cc9-c02932de428b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:01.314840 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144dc2a-18ce-4276-8cc9-c02932de428b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:01.333429 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-4144dc2a-18ce-4276-8cc9-c02932de428b None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:10:01.334649 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-4144dc2a-18ce-4276-8cc9-c02932de428b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:10:01.406418 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 506/1019] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:10:01 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 04:10:01.428695 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1af0da-1126-4221-8198-5f2cefcf4a21 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:01.540972 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1af0da-1126-4221-8198-5f2cefcf4a21 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:01.564409 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:01.576013 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0679b3-0437-4df0-9d28-1a872ed41b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:01.576640 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0679b3-0437-4df0-9d28-1a872ed41b8c None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:01.576922 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0679b3-0437-4df0-9d28-1a872ed41b8c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:01.577197 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0679b3-0437-4df0-9d28-1a872ed41b8c None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:01.577928 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0679b3-0437-4df0-9d28-1a872ed41b8c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:01.579318 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0679b3-0437-4df0-9d28-1a872ed41b8c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:01.602006 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 514/1020] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Jul 27 04:10:01 2022] POST /identity/v3/services => generated 242 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:10:01.834818 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ef710-1afd-4a77-9ad7-bd32d2243389 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:01.835165 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ef710-1afd-4a77-9ad7-bd32d2243389 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:01.835381 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ef710-1afd-4a77-9ad7-bd32d2243389 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:01.836871 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 507/1021] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:10:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:10:01.841796 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d0abf-4181-412b-897b-b705b6d85607 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:01.842117 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d0abf-4181-412b-897b-b705b6d85607 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:01.842391 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d0abf-4181-412b-897b-b705b6d85607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:01.860896 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-a78d0abf-4181-412b-897b-b705b6d85607 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:10:01.862124 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a78d0abf-4181-412b-897b-b705b6d85607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:10:01.937920 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 515/1022] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:10:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:10:01.961391 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ef710-1afd-4a77-9ad7-bd32d2243389 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:02.002566 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:02.067839 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ef710-1afd-4a77-9ad7-bd32d2243389 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:02.103229 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e127f-e266-4f9a-ac55-72fe3ed7b674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:02.103803 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e127f-e266-4f9a-ac55-72fe3ed7b674 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:02.104081 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e127f-e266-4f9a-ac55-72fe3ed7b674 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:02.104353 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e127f-e266-4f9a-ac55-72fe3ed7b674 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:02.114934 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e127f-e266-4f9a-ac55-72fe3ed7b674 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:10:02.119792 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-bc2e127f-e266-4f9a-ac55-72fe3ed7b674 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 04:10:02.119792 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:10:02.119792 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:10:02.119792 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:10:02.119792 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:10:02.119792 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:10:02.119792 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:10:02.119792 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:10:02.119792 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:10:02.119792 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:10:02.119792 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:10:02.119792 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:10:02.119792 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:10:02.119792 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:10:02.119792 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:10:02.119792 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:10:02.120710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:10:02.120710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:10:02.120710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:10:02.120710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:10:02.120710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:10:02.120710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:10:02.120710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:10:02.120710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:10:02.120710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:10:02.120710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:10:02.120710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:10:02.120710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:10:02.120710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:10:02.120710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:10:02.120710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:10:02.120710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:10:02.121625 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:10:02.121625 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:10:02.121625 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:10:02.121625 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:10:02.121625 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:10:02.121625 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:10:02.121625 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:10:02.121625 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:10:02.121625 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:10:02.121625 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:10:02.121625 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 04:10:02.121625 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:10:02.122878 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 508/1023] 10.222.0.46 () {56 vars in 1120 bytes} [Wed Jul 27 04:10:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:10:02.127791 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d0abf-4181-412b-897b-b705b6d85607 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:02.227818 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d0abf-4181-412b-897b-b705b6d85607 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:02.239602 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:02.257169 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9813c0f6-c2cf-44d5-aadf-299c60b43412 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:02.257449 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9813c0f6-c2cf-44d5-aadf-299c60b43412 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:02.257732 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9813c0f6-c2cf-44d5-aadf-299c60b43412 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:02.259255 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 509/1024] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:10:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 04:10:02.263995 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4757760-06e7-492a-8de3-4ca7a98a052f None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:02.264278 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4757760-06e7-492a-8de3-4ca7a98a052f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:02.264555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4757760-06e7-492a-8de3-4ca7a98a052f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:02.269058 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0c461-2fb0-4e26-b36e-2b007cba670f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:02.269886 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0c461-2fb0-4e26-b36e-2b007cba670f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:02.270343 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0c461-2fb0-4e26-b36e-2b007cba670f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:02.270763 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0c461-2fb0-4e26-b36e-2b007cba670f None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:02.271629 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b0c461-2fb0-4e26-b36e-2b007cba670f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:02.272101 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b0c461-2fb0-4e26-b36e-2b007cba670f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:02.273630 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b0c461-2fb0-4e26-b36e-2b007cba670f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:02.286560 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-c4757760-06e7-492a-8de3-4ca7a98a052f None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:10:02.286560 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c4757760-06e7-492a-8de3-4ca7a98a052f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:10:02.287156 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 516/1025] 10.222.0.46 () {56 vars in 1115 bytes} [Wed Jul 27 04:10:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:02.288211 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0c461-2fb0-4e26-b36e-2b007cba670f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:02.360317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 510/1026] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:10:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 04:10:02.384249 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4757760-06e7-492a-8de3-4ca7a98a052f None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:02.392734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0c461-2fb0-4e26-b36e-2b007cba670f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:02.412518 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:02.429024 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957afea-78a7-43ce-b660-8bfa6847af6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:02.429666 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957afea-78a7-43ce-b660-8bfa6847af6f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:02.429946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957afea-78a7-43ce-b660-8bfa6847af6f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:02.430247 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957afea-78a7-43ce-b660-8bfa6847af6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:02.443129 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9957afea-78a7-43ce-b660-8bfa6847af6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:02.444565 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9957afea-78a7-43ce-b660-8bfa6847af6f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:02.465962 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-9957afea-78a7-43ce-b660-8bfa6847af6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:10:02.465962 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:10:02.465962 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:10:02.465962 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:10:02.465962 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:10:02.465962 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:10:02.465962 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:10:02.465962 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:10:02.465962 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:10:02.465962 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:10:02.465962 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:10:02.465962 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:10:02.465962 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:10:02.465962 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:10:02.465962 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:10:02.466871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:10:02.466871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:10:02.466871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:10:02.466871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:10:02.466871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:10:02.466871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:10:02.466871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:10:02.466871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:10:02.466871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:10:02.466871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:10:02.466871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:10:02.466871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:10:02.466871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:10:02.466871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:10:02.466871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:10:02.466871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:10:02.466871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:10:02.467753 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:10:02.467753 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:10:02.467753 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:10:02.467753 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:10:02.467753 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 517/1027] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:10:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:10:02.473133 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957afea-78a7-43ce-b660-8bfa6847af6f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:02.490957 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4757760-06e7-492a-8de3-4ca7a98a052f None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:02.499085 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:02.526786 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0e9f50-82de-4bb0-8d9f-a4b180fc5d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:02.527433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e9f50-82de-4bb0-8d9f-a4b180fc5d05 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:02.527710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e9f50-82de-4bb0-8d9f-a4b180fc5d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:02.527982 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e9f50-82de-4bb0-8d9f-a4b180fc5d05 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:02.529182 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0e9f50-82de-4bb0-8d9f-a4b180fc5d05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:02.530703 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0e9f50-82de-4bb0-8d9f-a4b180fc5d05 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:02.532759 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 511/1028] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:10:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:10:02.538136 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0e9f50-82de-4bb0-8d9f-a4b180fc5d05 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:02.588030 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957afea-78a7-43ce-b660-8bfa6847af6f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:02.624299 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec2a3a-8587-405a-8de2-dfa8f6dc99c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:02.625846 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec2a3a-8587-405a-8de2-dfa8f6dc99c9 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:02.626157 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec2a3a-8587-405a-8de2-dfa8f6dc99c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:02.626432 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec2a3a-8587-405a-8de2-dfa8f6dc99c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:02.627480 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec2a3a-8587-405a-8de2-dfa8f6dc99c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:02.627946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec2a3a-8587-405a-8de2-dfa8f6dc99c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:02.629313 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec2a3a-8587-405a-8de2-dfa8f6dc99c9 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:02.641363 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 518/1029] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:10:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:10:02.644368 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:02.646945 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec2a3a-8587-405a-8de2-dfa8f6dc99c9 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:02.651434 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0e9f50-82de-4bb0-8d9f-a4b180fc5d05 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:02.689381 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702d136-3f7d-465c-bd72-f8253dfe829d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:02.690038 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702d136-3f7d-465c-bd72-f8253dfe829d None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:02.690326 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702d136-3f7d-465c-bd72-f8253dfe829d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:02.690598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702d136-3f7d-465c-bd72-f8253dfe829d None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:02.691290 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702d136-3f7d-465c-bd72-f8253dfe829d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:02.692780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702d136-3f7d-465c-bd72-f8253dfe829d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:02.717852 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 512/1030] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 27 04:10:02 2022] POST /identity/v3/users => generated 285 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:10:02.757578 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:02.777153 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec2a3a-8587-405a-8de2-dfa8f6dc99c9 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:02.816460 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65368beb-bca1-4b36-85dd-669ee128c89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:02.817132 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65368beb-bca1-4b36-85dd-669ee128c89f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:02.817419 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65368beb-bca1-4b36-85dd-669ee128c89f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:02.817697 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65368beb-bca1-4b36-85dd-669ee128c89f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:02.825805 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65368beb-bca1-4b36-85dd-669ee128c89f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:02.827312 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65368beb-bca1-4b36-85dd-669ee128c89f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:02.841123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-65368beb-bca1-4b36-85dd-669ee128c89f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 04:10:02.841123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:10:02.841123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:10:02.841123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:10:02.841123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:10:02.841123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:10:02.841123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:10:02.841123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:10:02.841123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:10:02.841123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:10:02.841123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:10:02.841123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:10:02.841123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:10:02.841123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:10:02.841123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:10:02.841123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:10:02.842457 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:10:02.842457 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:10:02.842457 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:10:02.842457 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:10:02.842457 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:10:02.842457 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:10:02.842457 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:10:02.842457 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:10:02.842457 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:10:02.842457 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:10:02.842457 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:10:02.842457 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:10:02.842457 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:10:02.842457 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:10:02.842457 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:10:02.842457 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:10:02.842457 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:10:02.842457 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:10:02.844425 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:10:02.844425 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:10:02.844425 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:10:02.844425 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:10:02.844425 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:10:02.844425 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:10:02.844425 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:10:02.844425 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:10:02.844425 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 04:10:02.844425 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:10:02.844425 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 519/1031] 10.222.0.46 () {56 vars in 1093 bytes} [Wed Jul 27 04:10:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:10:02.848950 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702d136-3f7d-465c-bd72-f8253dfe829d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:02.958371 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702d136-3f7d-465c-bd72-f8253dfe829d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:02.994453 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9944922-3bfc-43d7-8663-e6c7d85b62dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:02.995086 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9944922-3bfc-43d7-8663-e6c7d85b62dc None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:02.995376 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9944922-3bfc-43d7-8663-e6c7d85b62dc None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:02.995656 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9944922-3bfc-43d7-8663-e6c7d85b62dc None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:02.996464 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9944922-3bfc-43d7-8663-e6c7d85b62dc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:02.996776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9944922-3bfc-43d7-8663-e6c7d85b62dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:02.998242 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9944922-3bfc-43d7-8663-e6c7d85b62dc None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:03.016868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 513/1032] 10.222.0.46 () {56 vars in 1133 bytes} [Wed Jul 27 04:10:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:03.022591 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65368beb-bca1-4b36-85dd-669ee128c89f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:03.125229 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65368beb-bca1-4b36-85dd-669ee128c89f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:03.159430 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791f854-3f9e-48be-81e1-27baa2988c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:03.160059 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791f854-3f9e-48be-81e1-27baa2988c33 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:03.160346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791f854-3f9e-48be-81e1-27baa2988c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:03.160625 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791f854-3f9e-48be-81e1-27baa2988c33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:03.176064 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:03.231946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b791f854-3f9e-48be-81e1-27baa2988c33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:03.233310 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b791f854-3f9e-48be-81e1-27baa2988c33 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:03.344600 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 520/1033] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:10:03 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 27 04:10:03.346264 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9944922-3bfc-43d7-8663-e6c7d85b62dc None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:03.359198 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:03.447540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9944922-3bfc-43d7-8663-e6c7d85b62dc None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:03.488527 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1e6b1-7a45-4cb2-bd84-6a7dc7de9328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:03.489098 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1e6b1-7a45-4cb2-bd84-6a7dc7de9328 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:03.489380 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1e6b1-7a45-4cb2-bd84-6a7dc7de9328 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:03.489652 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1e6b1-7a45-4cb2-bd84-6a7dc7de9328 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:03.500198 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e1e6b1-7a45-4cb2-bd84-6a7dc7de9328 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:03.501658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e1e6b1-7a45-4cb2-bd84-6a7dc7de9328 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:03.517236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-92e1e6b1-7a45-4cb2-bd84-6a7dc7de9328 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:10:03.517236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:10:03.517236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:10:03.517236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:10:03.517236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:10:03.517236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:10:03.517236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:10:03.517236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:10:03.517236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:10:03.517236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:10:03.517236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:10:03.517236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:10:03.517236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:10:03.517236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:10:03.517236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:10:03.518559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:10:03.518559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:10:03.518559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:10:03.518559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:10:03.518559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:10:03.518559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:10:03.518559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:10:03.518559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:10:03.518559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:10:03.518559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:10:03.518559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:10:03.518559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:10:03.518559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:10:03.518559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:10:03.518559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:10:03.518559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:10:03.518559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:10:03.519568 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:10:03.519568 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:10:03.519568 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:10:03.519568 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:10:03.519568 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 514/1034] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:10:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:10:03.525736 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791f854-3f9e-48be-81e1-27baa2988c33 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:03.571820 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:03.633017 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791f854-3f9e-48be-81e1-27baa2988c33 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:03.640894 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d78cd-dc73-47ce-801b-0028ae668e67 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:03.641184 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d78cd-dc73-47ce-801b-0028ae668e67 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:03.641470 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d78cd-dc73-47ce-801b-0028ae668e67 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:03.642835 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 515/1035] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 04:10:03.647797 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80707044-f399-4cb1-969f-c6f946abda04 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:03.648322 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80707044-f399-4cb1-969f-c6f946abda04 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:03.648811 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80707044-f399-4cb1-969f-c6f946abda04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:03.666825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-80707044-f399-4cb1-969f-c6f946abda04 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:10:03.668308 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-80707044-f399-4cb1-969f-c6f946abda04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:10:03.670051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa788bf8-e04b-4b33-be26-4fbfba8d2a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:03.670673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa788bf8-e04b-4b33-be26-4fbfba8d2a11 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:03.670956 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa788bf8-e04b-4b33-be26-4fbfba8d2a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:03.671233 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa788bf8-e04b-4b33-be26-4fbfba8d2a11 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:03.671929 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa788bf8-e04b-4b33-be26-4fbfba8d2a11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:03.672232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa788bf8-e04b-4b33-be26-4fbfba8d2a11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:03.673609 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa788bf8-e04b-4b33-be26-4fbfba8d2a11 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:03.690897 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 521/1036] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:10:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:03.692653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa788bf8-e04b-4b33-be26-4fbfba8d2a11 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:03.728214 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:03.740027 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 516/1037] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:10:03 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 04:10:03.762171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80707044-f399-4cb1-969f-c6f946abda04 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:03.800570 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa788bf8-e04b-4b33-be26-4fbfba8d2a11 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:03.802917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:03.837472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66bde82-1720-4fc6-a221-89fe19ba1536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:03.838146 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bde82-1720-4fc6-a221-89fe19ba1536 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:03.838432 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bde82-1720-4fc6-a221-89fe19ba1536 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:03.838710 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bde82-1720-4fc6-a221-89fe19ba1536 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:03.845775 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66bde82-1720-4fc6-a221-89fe19ba1536 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:03.847247 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66bde82-1720-4fc6-a221-89fe19ba1536 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:03.858518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-f66bde82-1720-4fc6-a221-89fe19ba1536 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:10:03.858518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:10:03.858518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:10:03.858518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:10:03.858518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:10:03.858518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:10:03.858518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:10:03.858518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:10:03.858518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:10:03.858518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:10:03.858518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:10:03.858518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:10:03.858518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:10:03.858518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:10:03.858518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:10:03.859603 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:10:03.859603 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:10:03.859603 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:10:03.859603 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:10:03.859603 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:10:03.859603 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:10:03.859603 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:10:03.859603 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:10:03.859603 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:10:03.859603 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:10:03.859603 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:10:03.859603 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:10:03.859603 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:10:03.859603 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:10:03.859603 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:10:03.859603 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:10:03.859603 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:10:03.860518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:10:03.860518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:10:03.860518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:10:03.860518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:10:03.860518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:10:03.860518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:10:03.860518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:10:03.860518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:10:03.860518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 522/1038] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:10:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:10:03.866074 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66bde82-1720-4fc6-a221-89fe19ba1536 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:03.871885 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80707044-f399-4cb1-969f-c6f946abda04 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:03.908883 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa934f21-a0ef-4cf1-97af-e1e1ae7e3b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:03.909961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa934f21-a0ef-4cf1-97af-e1e1ae7e3b60 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:03.910280 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa934f21-a0ef-4cf1-97af-e1e1ae7e3b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:03.910558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa934f21-a0ef-4cf1-97af-e1e1ae7e3b60 None admin] PATH_INFO: `/v3/services/image` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:03.911200 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa934f21-a0ef-4cf1-97af-e1e1ae7e3b60 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:03.913230 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa934f21-a0ef-4cf1-97af-e1e1ae7e3b60 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:03.924097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-fa934f21-a0ef-4cf1-97af-e1e1ae7e3b60 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 04:10:03.924097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:10:03.924097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 04:10:03.924097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 04:10:03.924097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 04:10:03.924097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 04:10:03.924097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 04:10:03.924097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:10:03.924097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 04:10:03.924097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:10:03.924097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 04:10:03.924097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:10:03.924097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:10:03.924097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:10:03.924097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:10:03.924097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:10:03.924097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:10:03.925408 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:10:03.925408 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:10:03.925408 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:10:03.925408 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:10:03.925408 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:10:03.925408 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:10:03.925408 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 04:10:03.925408 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 04:10:03.925408 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 04:10:03.925408 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 04:10:03.925408 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:10:03.925408 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:10:03.925408 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:10:03.925408 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:10:03.925408 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:10:03.925408 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:10:03.926487 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:10:03.926487 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:10:03.926487 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:10:03.926487 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:10:03.926487 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:10:03.926487 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:10:03.926487 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:10:03.926487 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:10:03.926487 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:10:03.926487 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:10:03.926487 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 04:10:03.926487 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:10:03.926487 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 04:10:03.926487 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:10:03.930980 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 517/1039] 10.222.0.46 () {56 vars in 1105 bytes} [Wed Jul 27 04:10:03 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:10:03.931924 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa934f21-a0ef-4cf1-97af-e1e1ae7e3b60 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:03.988497 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66bde82-1720-4fc6-a221-89fe19ba1536 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:03.994287 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:04.024817 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f77fe98-355e-40fc-bf9e-247b0068ccfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:04.025448 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f77fe98-355e-40fc-bf9e-247b0068ccfa None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:04.025728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f77fe98-355e-40fc-bf9e-247b0068ccfa None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:04.026022 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f77fe98-355e-40fc-bf9e-247b0068ccfa None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:04.026778 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f77fe98-355e-40fc-bf9e-247b0068ccfa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:04.027093 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f77fe98-355e-40fc-bf9e-247b0068ccfa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:04.028511 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f77fe98-355e-40fc-bf9e-247b0068ccfa None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:04.041424 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 523/1040] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:10:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:10:04.047242 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f77fe98-355e-40fc-bf9e-247b0068ccfa None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:04.049744 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa934f21-a0ef-4cf1-97af-e1e1ae7e3b60 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:04.072055 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:04.085788 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae3c297-51db-4e4c-8ac9-852d8b459b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:04.086442 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3c297-51db-4e4c-8ac9-852d8b459b98 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:04.086725 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3c297-51db-4e4c-8ac9-852d8b459b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:04.086999 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3c297-51db-4e4c-8ac9-852d8b459b98 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:04.087682 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae3c297-51db-4e4c-8ac9-852d8b459b98 None admin] RBAC: Adding query filter params (name=image) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:04.087989 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae3c297-51db-4e4c-8ac9-852d8b459b98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:04.089361 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae3c297-51db-4e4c-8ac9-852d8b459b98 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:04.095193 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 518/1041] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:10:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:04.100649 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae3c297-51db-4e4c-8ac9-852d8b459b98 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:04.155144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f77fe98-355e-40fc-bf9e-247b0068ccfa None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:04.195909 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd85e47a-2340-462e-9f81-196e6c22f8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:04.196542 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85e47a-2340-462e-9f81-196e6c22f8f4 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:04.196828 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85e47a-2340-462e-9f81-196e6c22f8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:04.197107 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85e47a-2340-462e-9f81-196e6c22f8f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:04.197906 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd85e47a-2340-462e-9f81-196e6c22f8f4 None admin] RBAC: Adding query filter params (role.id=3298581080b940c1ab0a41e5bbc8953f, scope.project.id=e646d320aa5a4f78bfccaf941cf46478, user.id=9dc9ea8f89c14a2a8e3272f750647dfd) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:04.198254 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd85e47a-2340-462e-9f81-196e6c22f8f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:04.199678 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd85e47a-2340-462e-9f81-196e6c22f8f4 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:04.205741 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 524/1042] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:10:04 2022] GET /identity/v3/role_assignments?user.id=9dc9ea8f89c14a2a8e3272f750647dfd&scope.project.id=e646d320aa5a4f78bfccaf941cf46478&role.id=3298581080b940c1ab0a41e5bbc8953f => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:10:04.208155 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae3c297-51db-4e4c-8ac9-852d8b459b98 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:04.233191 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:04.251606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7422623c-52f9-4899-917c-3164165f2732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:04.252177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422623c-52f9-4899-917c-3164165f2732 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:04.252459 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422623c-52f9-4899-917c-3164165f2732 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:04.252731 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422623c-52f9-4899-917c-3164165f2732 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:04.253448 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7422623c-52f9-4899-917c-3164165f2732 None admin] RBAC: Adding query filter params (type=image) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:04.253755 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7422623c-52f9-4899-917c-3164165f2732 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:04.255218 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7422623c-52f9-4899-917c-3164165f2732 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:04.266077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 519/1043] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:10:04 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:10:04.267513 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd85e47a-2340-462e-9f81-196e6c22f8f4 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:04.368921 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:04.383452 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd85e47a-2340-462e-9f81-196e6c22f8f4 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:04.419164 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c33c8-0781-4c98-9034-a9e6bcd2b7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:04.419792 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c33c8-0781-4c98-9034-a9e6bcd2b7a6 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:04.420079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c33c8-0781-4c98-9034-a9e6bcd2b7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:04.420358 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c33c8-0781-4c98-9034-a9e6bcd2b7a6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:04.421126 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6c33c8-0781-4c98-9034-a9e6bcd2b7a6 None admin] RBAC: Adding query filter params (interface=public, service_id=88a85953562f4386a63ab9f468f2784f, region_id=RegionOne) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:04.421497 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6c33c8-0781-4c98-9034-a9e6bcd2b7a6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:04.423483 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6c33c8-0781-4c98-9034-a9e6bcd2b7a6 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:04.428965 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 525/1044] 10.222.0.46 () {56 vars in 1251 bytes} [Wed Jul 27 04:10:04 2022] GET /identity/v3/endpoints?service_id=88a85953562f4386a63ab9f468f2784f&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:10:04.434328 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7422623c-52f9-4899-917c-3164165f2732 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:04.547600 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7422623c-52f9-4899-917c-3164165f2732 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:04.583242 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd7205-5cc4-4f2e-bd8d-376632e82674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:04.583869 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd7205-5cc4-4f2e-bd8d-376632e82674 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:04.584154 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd7205-5cc4-4f2e-bd8d-376632e82674 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:04.584432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd7205-5cc4-4f2e-bd8d-376632e82674 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:04.585096 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bd7205-5cc4-4f2e-bd8d-376632e82674 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:04.586498 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bd7205-5cc4-4f2e-bd8d-376632e82674 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:04.602016 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 520/1045] 10.222.0.46 () {56 vars in 1087 bytes} [Wed Jul 27 04:10:04 2022] GET /identity/v3/services => generated 1957 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:10:04.834688 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e29ab-1bb4-4a3e-abd3-75423878e9ee None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:04.835186 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e29ab-1bb4-4a3e-abd3-75423878e9ee None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:04.835295 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e29ab-1bb4-4a3e-abd3-75423878e9ee None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:04.836710 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 526/1046] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:10:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:10:04.841745 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7367747-2a8a-429a-a69a-29dc93f29075 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:04.842063 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7367747-2a8a-429a-a69a-29dc93f29075 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:04.842343 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7367747-2a8a-429a-a69a-29dc93f29075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:04.861267 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-e7367747-2a8a-429a-a69a-29dc93f29075 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:10:04.862596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e7367747-2a8a-429a-a69a-29dc93f29075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:10:04.908228 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:04.933220 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 521/1047] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:10:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:10:04.956597 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e29ab-1bb4-4a3e-abd3-75423878e9ee None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:05.057841 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e29ab-1bb4-4a3e-abd3-75423878e9ee None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:05.093104 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c2aa89-511c-4563-96a9-73df1207e0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:05.093752 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c2aa89-511c-4563-96a9-73df1207e0c5 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:05.094188 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c2aa89-511c-4563-96a9-73df1207e0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:05.094514 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c2aa89-511c-4563-96a9-73df1207e0c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:05.104608 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c2aa89-511c-4563-96a9-73df1207e0c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:10:05.110155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-89c2aa89-511c-4563-96a9-73df1207e0c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:10:05.110155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:10:05.110155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:10:05.110155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:10:05.110155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:10:05.110155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:10:05.110155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:10:05.110155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:10:05.110155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:10:05.110155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:10:05.110155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:10:05.110155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:10:05.110155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:10:05.110155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:10:05.110155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:10:05.110155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:10:05.111051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:10:05.111051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:10:05.111051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:10:05.111051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:10:05.111051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:10:05.111051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:10:05.111051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:10:05.111051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:10:05.111051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:10:05.111051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:10:05.111051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:10:05.111051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:10:05.111051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:10:05.111051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:10:05.111051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:10:05.111051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:10:05.111909 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:10:05.111909 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:10:05.111909 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:10:05.111909 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:10:05.111909 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:10:05.111909 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:10:05.111909 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:10:05.111909 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:10:05.111909 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:10:05.111909 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:10:05.111909 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:10:05.111909 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:10:05.114378 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 527/1048] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:10:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:10:05.117396 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7367747-2a8a-429a-a69a-29dc93f29075 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:05.216514 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7367747-2a8a-429a-a69a-29dc93f29075 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:05.250915 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6668155b-8133-4710-9f21-b2c2ae1215c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:05.251530 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6668155b-8133-4710-9f21-b2c2ae1215c0 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:05.251806 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6668155b-8133-4710-9f21-b2c2ae1215c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:05.252076 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6668155b-8133-4710-9f21-b2c2ae1215c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:05.252794 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6668155b-8133-4710-9f21-b2c2ae1215c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:05.253095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6668155b-8133-4710-9f21-b2c2ae1215c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:05.254492 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6668155b-8133-4710-9f21-b2c2ae1215c0 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:05.268031 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 522/1049] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:10:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:05.269218 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c2aa89-511c-4563-96a9-73df1207e0c5 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:05.285677 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:05.376492 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c2aa89-511c-4563-96a9-73df1207e0c5 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:05.410163 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f570b6-7a19-469b-af3e-0abd9d9a26c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:05.410803 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f570b6-7a19-469b-af3e-0abd9d9a26c1 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:05.411087 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f570b6-7a19-469b-af3e-0abd9d9a26c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:05.411360 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f570b6-7a19-469b-af3e-0abd9d9a26c1 None admin] PATH_INFO: `/v3/users/3281369f31b54b32804554ba9f67673b` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:05.424513 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f570b6-7a19-469b-af3e-0abd9d9a26c1 None admin] RBAC: Authorizing `identity:get_user(user_id=3281369f31b54b32804554ba9f67673b)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:05.425938 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f570b6-7a19-469b-af3e-0abd9d9a26c1 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:05.427727 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 528/1050] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:10:05 2022] GET /identity/v3/users/3281369f31b54b32804554ba9f67673b => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:05.433392 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6668155b-8133-4710-9f21-b2c2ae1215c0 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:05.483201 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:05.541950 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6668155b-8133-4710-9f21-b2c2ae1215c0 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:05.575590 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb63c5c-f3bd-4083-be07-8ff46691e815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:05.576205 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb63c5c-f3bd-4083-be07-8ff46691e815 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:05.576480 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb63c5c-f3bd-4083-be07-8ff46691e815 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:05.576750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb63c5c-f3bd-4083-be07-8ff46691e815 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:05.644129 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb63c5c-f3bd-4083-be07-8ff46691e815 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:05.645516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb63c5c-f3bd-4083-be07-8ff46691e815 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:05.692211 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:05.757114 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 523/1051] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:10:05 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 04:10:05.762801 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f570b6-7a19-469b-af3e-0abd9d9a26c1 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:05.834579 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:05.866830 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f570b6-7a19-469b-af3e-0abd9d9a26c1 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:05.900543 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19002a6c-d83c-40f7-86e5-e1b6e3a9803a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:05.901124 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19002a6c-d83c-40f7-86e5-e1b6e3a9803a None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:05.901387 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19002a6c-d83c-40f7-86e5-e1b6e3a9803a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:05.901657 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19002a6c-d83c-40f7-86e5-e1b6e3a9803a None admin] PATH_INFO: `/v3/projects/b4e2148743504b999b0e30a1fb3e97c7` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:05.902840 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19002a6c-d83c-40f7-86e5-e1b6e3a9803a None admin] RBAC: Authorizing `identity:get_project(project_id=b4e2148743504b999b0e30a1fb3e97c7)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:05.904264 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19002a6c-d83c-40f7-86e5-e1b6e3a9803a None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:05.906626 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 529/1052] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:10:05 2022] GET /identity/v3/projects/b4e2148743504b999b0e30a1fb3e97c7 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:05.912700 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb63c5c-f3bd-4083-be07-8ff46691e815 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:06.014475 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb63c5c-f3bd-4083-be07-8ff46691e815 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:06.049028 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d5024-984d-4feb-87e0-5c0f2bc86c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:06.049657 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d5024-984d-4feb-87e0-5c0f2bc86c8e None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:06.049941 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d5024-984d-4feb-87e0-5c0f2bc86c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:06.050249 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d5024-984d-4feb-87e0-5c0f2bc86c8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:06.051046 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d5024-984d-4feb-87e0-5c0f2bc86c8e None admin] RBAC: Adding query filter params (role.id=cf771a8e94ee46149eb081347fea6d67, scope.project.id=b4e2148743504b999b0e30a1fb3e97c7, user.id=3281369f31b54b32804554ba9f67673b) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:06.051358 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d5024-984d-4feb-87e0-5c0f2bc86c8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:06.052805 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d5024-984d-4feb-87e0-5c0f2bc86c8e None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:06.066074 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 524/1053] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:10:05 2022] GET /identity/v3/role_assignments?user.id=3281369f31b54b32804554ba9f67673b&scope.project.id=b4e2148743504b999b0e30a1fb3e97c7&role.id=cf771a8e94ee46149eb081347fea6d67 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:06.208263 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90e40c-a353-4e56-bbc2-13f4eb23d9e2 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:06.208458 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90e40c-a353-4e56-bbc2-13f4eb23d9e2 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:06.208733 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90e40c-a353-4e56-bbc2-13f4eb23d9e2 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:06.210478 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 530/1054] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:10:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:10:06.215498 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09392b42-0e37-4564-bca0-1164ddee9cb8 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:06.215781 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09392b42-0e37-4564-bca0-1164ddee9cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:06.216059 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09392b42-0e37-4564-bca0-1164ddee9cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:06.224352 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:06.236678 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-09392b42-0e37-4564-bca0-1164ddee9cb8 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:10:06.237868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-09392b42-0e37-4564-bca0-1164ddee9cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:10:06.305535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 525/1055] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:10:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 04:10:06.328275 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90e40c-a353-4e56-bbc2-13f4eb23d9e2 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:06.430728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90e40c-a353-4e56-bbc2-13f4eb23d9e2 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:06.474567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d090134f-1fe1-41b7-abcf-38a6bd187795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:06.475144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d090134f-1fe1-41b7-abcf-38a6bd187795 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:06.475425 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d090134f-1fe1-41b7-abcf-38a6bd187795 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:06.475700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d090134f-1fe1-41b7-abcf-38a6bd187795 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:06.482798 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:06.488052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d090134f-1fe1-41b7-abcf-38a6bd187795 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:10:06.492946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-d090134f-1fe1-41b7-abcf-38a6bd187795 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 04:10:06.492946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:10:06.492946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:10:06.492946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:10:06.492946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:10:06.492946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:10:06.492946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:10:06.492946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:10:06.492946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:10:06.492946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:10:06.492946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:10:06.492946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:10:06.492946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:10:06.492946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:10:06.492946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:10:06.492946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:10:06.493875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:10:06.493875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:10:06.493875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:10:06.493875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:10:06.493875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:10:06.493875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:10:06.493875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:10:06.493875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:10:06.493875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:10:06.493875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:10:06.493875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:10:06.493875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:10:06.493875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:10:06.493875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:10:06.493875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:10:06.493875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:10:06.499637 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:10:06.499637 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:10:06.499637 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:10:06.499637 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:10:06.499637 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:10:06.499637 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:10:06.499637 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:10:06.499637 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:10:06.499637 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:10:06.499637 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:10:06.499637 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 04:10:06.499637 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:10:06.499637 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 531/1056] 10.222.0.46 () {56 vars in 1120 bytes} [Wed Jul 27 04:10:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:10:06.502157 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09392b42-0e37-4564-bca0-1164ddee9cb8 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:06.608441 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09392b42-0e37-4564-bca0-1164ddee9cb8 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:06.627949 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e56fd-73d7-489f-abd6-68709c680e2b None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:06.628319 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e56fd-73d7-489f-abd6-68709c680e2b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:06.628684 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e56fd-73d7-489f-abd6-68709c680e2b None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:06.630307 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 532/1057] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:10:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 04:10:06.634931 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e89ea0-c113-4119-b8f8-ebfd9bf8c5a1 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:06.635217 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e89ea0-c113-4119-b8f8-ebfd9bf8c5a1 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:06.635595 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e89ea0-c113-4119-b8f8-ebfd9bf8c5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:06.647396 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856d202-b505-492f-8697-22e04d3dec13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:06.648026 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856d202-b505-492f-8697-22e04d3dec13 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:06.648310 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856d202-b505-492f-8697-22e04d3dec13 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:06.648587 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856d202-b505-492f-8697-22e04d3dec13 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:06.649317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e856d202-b505-492f-8697-22e04d3dec13 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:06.649622 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e856d202-b505-492f-8697-22e04d3dec13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:06.651019 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e856d202-b505-492f-8697-22e04d3dec13 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:06.652558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:06.653029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-06e89ea0-c113-4119-b8f8-ebfd9bf8c5a1 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:10:06.654392 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-06e89ea0-c113-4119-b8f8-ebfd9bf8c5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:10:06.661639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 526/1058] 10.222.0.46 () {56 vars in 1115 bytes} [Wed Jul 27 04:10:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:06.668152 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856d202-b505-492f-8697-22e04d3dec13 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:06.734585 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 533/1059] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:10:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 04:10:06.752933 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e89ea0-c113-4119-b8f8-ebfd9bf8c5a1 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:06.778483 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856d202-b505-492f-8697-22e04d3dec13 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:06.818277 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e16ff95-abbd-47c5-ae2e-de8bb5546883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:06.818931 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16ff95-abbd-47c5-ae2e-de8bb5546883 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:06.819213 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16ff95-abbd-47c5-ae2e-de8bb5546883 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:06.819488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16ff95-abbd-47c5-ae2e-de8bb5546883 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:06.830765 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e16ff95-abbd-47c5-ae2e-de8bb5546883 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:06.832230 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e16ff95-abbd-47c5-ae2e-de8bb5546883 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:06.847520 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-8e16ff95-abbd-47c5-ae2e-de8bb5546883 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:10:06.847520 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:10:06.847520 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:10:06.847520 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:10:06.847520 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:10:06.847520 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:10:06.847520 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:10:06.847520 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:10:06.847520 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:10:06.847520 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:10:06.847520 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:10:06.847520 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:10:06.847520 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:10:06.847520 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:10:06.847520 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:10:06.848432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:10:06.848432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:10:06.848432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:10:06.848432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:10:06.848432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:10:06.848432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:10:06.848432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:10:06.848432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:10:06.848432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:10:06.848432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:10:06.848432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:10:06.848432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:10:06.848432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:10:06.848432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:10:06.848432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:10:06.848432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:10:06.848432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:10:06.849440 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:10:06.849440 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:10:06.849440 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:10:06.849440 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:10:06.849440 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 527/1060] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:10:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:10:06.854370 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e16ff95-abbd-47c5-ae2e-de8bb5546883 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:06.861502 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e89ea0-c113-4119-b8f8-ebfd9bf8c5a1 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:06.893082 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:06.902288 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f613af-417f-46e0-8464-8f706c26d2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:06.903068 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f613af-417f-46e0-8464-8f706c26d2e9 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:06.903370 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f613af-417f-46e0-8464-8f706c26d2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:06.903645 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f613af-417f-46e0-8464-8f706c26d2e9 None admin] PATH_INFO: `/v3/services/image` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:06.904640 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f613af-417f-46e0-8464-8f706c26d2e9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:06.906203 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f613af-417f-46e0-8464-8f706c26d2e9 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:06.906845 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:06.915583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-85f613af-417f-46e0-8464-8f706c26d2e9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 04:10:06.915583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:10:06.915583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 04:10:06.915583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 04:10:06.915583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 04:10:06.915583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 04:10:06.915583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 04:10:06.915583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:10:06.915583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 04:10:06.915583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:10:06.915583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 04:10:06.915583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:10:06.915583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:10:06.915583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:10:06.915583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:10:06.915583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:10:06.915583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:10:06.916574 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:10:06.916574 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:10:06.916574 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:10:06.916574 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:10:06.916574 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:10:06.916574 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:10:06.916574 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 04:10:06.916574 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 04:10:06.916574 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 04:10:06.916574 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 04:10:06.916574 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:10:06.916574 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:10:06.916574 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:10:06.916574 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:10:06.916574 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:10:06.916574 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:10:06.917408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:10:06.917408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:10:06.917408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:10:06.917408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:10:06.917408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:10:06.917408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:10:06.917408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:10:06.917408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:10:06.917408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:10:06.917408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:10:06.917408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 04:10:06.917408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:10:06.917408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 04:10:06.917408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:10:06.917408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 534/1061] 10.222.0.46 () {56 vars in 1105 bytes} [Wed Jul 27 04:10:06 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:10:06.922874 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f613af-417f-46e0-8464-8f706c26d2e9 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:06.965072 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e16ff95-abbd-47c5-ae2e-de8bb5546883 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:07.000577 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7a020-d987-43b4-8d6c-6ab188cb791d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:07.001179 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7a020-d987-43b4-8d6c-6ab188cb791d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:07.001461 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7a020-d987-43b4-8d6c-6ab188cb791d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:07.001729 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7a020-d987-43b4-8d6c-6ab188cb791d None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:07.002474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7a020-d987-43b4-8d6c-6ab188cb791d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:07.002775 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7a020-d987-43b4-8d6c-6ab188cb791d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:07.004144 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7a020-d987-43b4-8d6c-6ab188cb791d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:07.016683 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 528/1062] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:10:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:10:07.021929 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7a020-d987-43b4-8d6c-6ab188cb791d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:07.029583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f613af-417f-46e0-8464-8f706c26d2e9 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:07.069296 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b8137-64a1-4ec9-8cb5-f47df340f80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:07.070152 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b8137-64a1-4ec9-8cb5-f47df340f80c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:07.070436 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b8137-64a1-4ec9-8cb5-f47df340f80c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:07.071108 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b8137-64a1-4ec9-8cb5-f47df340f80c None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:07.072082 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b8137-64a1-4ec9-8cb5-f47df340f80c None admin] RBAC: Adding query filter params (name=image) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:07.072396 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b8137-64a1-4ec9-8cb5-f47df340f80c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:07.074228 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b8137-64a1-4ec9-8cb5-f47df340f80c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:07.085294 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 535/1063] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:10:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:10:07.086852 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b8137-64a1-4ec9-8cb5-f47df340f80c None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:07.128325 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7a020-d987-43b4-8d6c-6ab188cb791d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:07.138114 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:07.148156 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:07.167429 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed23e1b-5354-4032-849d-fa24d9c45fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:07.168085 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed23e1b-5354-4032-849d-fa24d9c45fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:07.168362 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed23e1b-5354-4032-849d-fa24d9c45fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:07.168631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed23e1b-5354-4032-849d-fa24d9c45fe9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:07.178249 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed23e1b-5354-4032-849d-fa24d9c45fe9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:07.179684 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed23e1b-5354-4032-849d-fa24d9c45fe9 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:07.193114 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-fed23e1b-5354-4032-849d-fa24d9c45fe9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 04:10:07.193114 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:10:07.193114 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:10:07.193114 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:10:07.193114 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:10:07.193114 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:10:07.193114 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:10:07.193114 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:10:07.193114 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:10:07.193114 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:10:07.193114 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:10:07.193114 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:10:07.193114 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:10:07.193114 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:10:07.193114 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:10:07.193114 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:10:07.194249 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:10:07.194249 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:10:07.194249 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:10:07.194249 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:10:07.194249 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:10:07.194249 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:10:07.194249 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:10:07.194249 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:10:07.194249 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:10:07.194249 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:10:07.194249 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:10:07.194249 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:10:07.194249 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:10:07.194249 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:10:07.194249 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:10:07.194249 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:10:07.194249 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:10:07.194249 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:10:07.195257 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:10:07.195257 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:10:07.195257 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:10:07.195257 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:10:07.195257 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:10:07.195257 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:10:07.195257 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:10:07.195257 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:10:07.195257 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 04:10:07.195257 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:10:07.195257 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 529/1064] 10.222.0.46 () {56 vars in 1093 bytes} [Wed Jul 27 04:10:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:10:07.199925 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed23e1b-5354-4032-849d-fa24d9c45fe9 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:07.203435 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b8137-64a1-4ec9-8cb5-f47df340f80c None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:07.239230 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb1fd4-1f3f-4fb9-b5a5-2a54b2b0c7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:07.239957 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb1fd4-1f3f-4fb9-b5a5-2a54b2b0c7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:07.240247 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb1fd4-1f3f-4fb9-b5a5-2a54b2b0c7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:07.240527 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb1fd4-1f3f-4fb9-b5a5-2a54b2b0c7e0 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:07.241217 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb1fd4-1f3f-4fb9-b5a5-2a54b2b0c7e0 None admin] RBAC: Adding query filter params (type=image) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:07.241524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb1fd4-1f3f-4fb9-b5a5-2a54b2b0c7e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:07.242919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb1fd4-1f3f-4fb9-b5a5-2a54b2b0c7e0 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:07.248083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 536/1065] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:10:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:07.253929 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb1fd4-1f3f-4fb9-b5a5-2a54b2b0c7e0 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:07.309478 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed23e1b-5354-4032-849d-fa24d9c45fe9 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:07.346378 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474f68a2-0bef-4cfe-a2a2-572cc8cfabc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:07.347113 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f68a2-0bef-4cfe-a2a2-572cc8cfabc5 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:07.347390 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f68a2-0bef-4cfe-a2a2-572cc8cfabc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:07.347779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f68a2-0bef-4cfe-a2a2-572cc8cfabc5 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:07.348577 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474f68a2-0bef-4cfe-a2a2-572cc8cfabc5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:07.348898 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474f68a2-0bef-4cfe-a2a2-572cc8cfabc5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:07.350437 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474f68a2-0bef-4cfe-a2a2-572cc8cfabc5 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:07.365071 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb1fd4-1f3f-4fb9-b5a5-2a54b2b0c7e0 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:07.368876 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 530/1066] 10.222.0.46 () {56 vars in 1133 bytes} [Wed Jul 27 04:10:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:07.374190 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474f68a2-0bef-4cfe-a2a2-572cc8cfabc5 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:07.387373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:07.401605 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:07.405117 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446fb72-187a-4d7f-8272-8131d34b2d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:07.406248 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446fb72-187a-4d7f-8272-8131d34b2d5b None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:07.406524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446fb72-187a-4d7f-8272-8131d34b2d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:07.406794 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446fb72-187a-4d7f-8272-8131d34b2d5b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:07.407760 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1446fb72-187a-4d7f-8272-8131d34b2d5b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:07.409118 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1446fb72-187a-4d7f-8272-8131d34b2d5b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:07.440769 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 537/1067] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:10:07 2022] POST /identity/v3/endpoints => generated 327 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:10:07.487706 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474f68a2-0bef-4cfe-a2a2-572cc8cfabc5 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:07.533908 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f6c99-894b-40a0-b9dd-f996b589958d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:07.534512 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f6c99-894b-40a0-b9dd-f996b589958d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:07.534790 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f6c99-894b-40a0-b9dd-f996b589958d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:07.535060 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f6c99-894b-40a0-b9dd-f996b589958d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:07.546446 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f6c99-894b-40a0-b9dd-f996b589958d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:07.547913 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f6c99-894b-40a0-b9dd-f996b589958d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:07.564900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-687f6c99-894b-40a0-b9dd-f996b589958d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:10:07.564900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:10:07.564900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:10:07.564900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:10:07.564900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:10:07.564900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:10:07.564900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:10:07.564900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:10:07.564900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:10:07.564900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:10:07.564900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:10:07.564900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:10:07.564900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:10:07.564900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:10:07.564900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:10:07.565958 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:10:07.565958 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:10:07.565958 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:10:07.565958 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:10:07.565958 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:10:07.565958 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:10:07.565958 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:10:07.565958 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:10:07.565958 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:10:07.565958 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:10:07.565958 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:10:07.565958 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:10:07.565958 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:10:07.565958 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:10:07.565958 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:10:07.565958 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:10:07.565958 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:10:07.566913 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:10:07.566913 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:10:07.566913 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:10:07.566913 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:10:07.566913 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 531/1068] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:10:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:10:07.575309 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446fb72-187a-4d7f-8272-8131d34b2d5b None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:07.678710 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446fb72-187a-4d7f-8272-8131d34b2d5b None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:07.713222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df2063-e150-49e1-96a0-2d07d300a7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:07.713847 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df2063-e150-49e1-96a0-2d07d300a7f1 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:07.714157 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df2063-e150-49e1-96a0-2d07d300a7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:07.714431 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df2063-e150-49e1-96a0-2d07d300a7f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:07.715122 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df2063-e150-49e1-96a0-2d07d300a7f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:07.715423 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df2063-e150-49e1-96a0-2d07d300a7f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:07.716768 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df2063-e150-49e1-96a0-2d07d300a7f1 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:07.728702 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 538/1069] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:10:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:07.735135 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f6c99-894b-40a0-b9dd-f996b589958d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:07.818533 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:07.845330 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f6c99-894b-40a0-b9dd-f996b589958d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:07.880005 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211280b-931e-4f41-9e46-b65c59f3b8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:07.880659 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211280b-931e-4f41-9e46-b65c59f3b8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:07.880942 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211280b-931e-4f41-9e46-b65c59f3b8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:07.881216 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211280b-931e-4f41-9e46-b65c59f3b8f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:07.888014 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d211280b-931e-4f41-9e46-b65c59f3b8f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:07.889474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d211280b-931e-4f41-9e46-b65c59f3b8f7 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:07.901376 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-d211280b-931e-4f41-9e46-b65c59f3b8f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:10:07.901376 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:10:07.901376 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:10:07.901376 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:10:07.901376 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:10:07.901376 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:10:07.901376 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:10:07.901376 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:10:07.901376 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:10:07.901376 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:10:07.901376 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:10:07.901376 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:10:07.901376 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:10:07.901376 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:10:07.901376 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:10:07.902507 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:10:07.902507 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:10:07.902507 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:10:07.902507 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:10:07.902507 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:10:07.902507 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:10:07.902507 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:10:07.902507 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:10:07.902507 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:10:07.902507 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:10:07.902507 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:10:07.902507 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:10:07.902507 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:10:07.902507 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:10:07.902507 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:10:07.902507 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:10:07.902507 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:10:07.903416 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:10:07.903416 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:10:07.903416 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:10:07.903416 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:10:07.903416 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:10:07.903416 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:10:07.903416 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:10:07.903416 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:10:07.907373 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 532/1070] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:10:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:10:07.908364 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df2063-e150-49e1-96a0-2d07d300a7f1 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:07.945117 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:08.013083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df2063-e150-49e1-96a0-2d07d300a7f1 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:08.048795 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552446d-f032-4ecf-a50b-8ebbfa0dcd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:08.049420 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552446d-f032-4ecf-a50b-8ebbfa0dcd83 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:08.049705 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552446d-f032-4ecf-a50b-8ebbfa0dcd83 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:08.049998 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552446d-f032-4ecf-a50b-8ebbfa0dcd83 None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:08.050771 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6552446d-f032-4ecf-a50b-8ebbfa0dcd83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:08.051082 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6552446d-f032-4ecf-a50b-8ebbfa0dcd83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:08.052503 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6552446d-f032-4ecf-a50b-8ebbfa0dcd83 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:08.064592 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 539/1071] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:10:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:08.068607 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9085031-4695-4cf4-bb1d-17977e762df5 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:08.068825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9085031-4695-4cf4-bb1d-17977e762df5 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:08.069104 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9085031-4695-4cf4-bb1d-17977e762df5 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:08.070289 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552446d-f032-4ecf-a50b-8ebbfa0dcd83 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:08.070763 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 533/1072] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:10:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 04:10:08.075704 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849968a5-c1dd-4451-8aa3-ddab4391a501 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:08.075984 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849968a5-c1dd-4451-8aa3-ddab4391a501 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:08.076257 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849968a5-c1dd-4451-8aa3-ddab4391a501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:08.095977 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-849968a5-c1dd-4451-8aa3-ddab4391a501 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:10:08.097174 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-849968a5-c1dd-4451-8aa3-ddab4391a501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:10:08.170015 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 534/1073] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:10:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 04:10:08.174911 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:08.181011 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552446d-f032-4ecf-a50b-8ebbfa0dcd83 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:08.189897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849968a5-c1dd-4451-8aa3-ddab4391a501 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:08.218066 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763b957-0cd0-4798-9e99-b6c9d3acd4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:08.219623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763b957-0cd0-4798-9e99-b6c9d3acd4a8 None admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:08.219958 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763b957-0cd0-4798-9e99-b6c9d3acd4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:08.220213 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763b957-0cd0-4798-9e99-b6c9d3acd4a8 None admin] PATH_INFO: `/v3/projects/e646d320aa5a4f78bfccaf941cf46478/users/9dc9ea8f89c14a2a8e3272f750647dfd/roles/3298581080b940c1ab0a41e5bbc8953f` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:08.222528 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c763b957-0cd0-4798-9e99-b6c9d3acd4a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=e646d320aa5a4f78bfccaf941cf46478, user_id=9dc9ea8f89c14a2a8e3272f750647dfd, role_id=3298581080b940c1ab0a41e5bbc8953f)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:08.223990 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c763b957-0cd0-4798-9e99-b6c9d3acd4a8 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:08.239935 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 540/1074] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 27 04:10:08 2022] PUT /identity/v3/projects/e646d320aa5a4f78bfccaf941cf46478/users/9dc9ea8f89c14a2a8e3272f750647dfd/roles/3298581080b940c1ab0a41e5bbc8953f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:10:08.257748 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:08.301437 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849968a5-c1dd-4451-8aa3-ddab4391a501 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:08.339909 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e25ba-ea14-44b1-98a7-bfce0db5aa5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:08.340579 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e25ba-ea14-44b1-98a7-bfce0db5aa5a None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:08.340865 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e25ba-ea14-44b1-98a7-bfce0db5aa5a None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:08.341141 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e25ba-ea14-44b1-98a7-bfce0db5aa5a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:08.353530 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e25ba-ea14-44b1-98a7-bfce0db5aa5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:10:08.358596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-d74e25ba-ea14-44b1-98a7-bfce0db5aa5a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:10:08.358596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:10:08.358596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:10:08.358596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:10:08.358596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:10:08.358596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:10:08.358596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:10:08.358596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:10:08.358596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:10:08.358596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:10:08.358596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:10:08.358596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:10:08.358596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:10:08.358596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:10:08.358596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:10:08.358596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:10:08.360433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:10:08.360433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:10:08.360433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:10:08.360433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:10:08.360433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:10:08.360433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:10:08.360433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:10:08.360433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:10:08.360433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:10:08.360433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:10:08.360433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:10:08.360433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:10:08.360433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:10:08.360433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:10:08.360433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:10:08.360433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:10:08.361307 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:10:08.361307 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:10:08.361307 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:10:08.361307 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:10:08.361307 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:10:08.361307 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:10:08.361307 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:10:08.361307 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:10:08.361307 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:10:08.361307 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:10:08.361307 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:10:08.361307 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:10:08.361307 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 535/1075] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:10:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:10:08.366508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763b957-0cd0-4798-9e99-b6c9d3acd4a8 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:08.484341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763b957-0cd0-4798-9e99-b6c9d3acd4a8 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:08.519870 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f729f-a885-42c6-b42f-6a950c0aac84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:08.520831 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f729f-a885-42c6-b42f-6a950c0aac84 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:08.521312 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f729f-a885-42c6-b42f-6a950c0aac84 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:08.521787 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f729f-a885-42c6-b42f-6a950c0aac84 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:08.522728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82f729f-a885-42c6-b42f-6a950c0aac84 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:08.523255 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82f729f-a885-42c6-b42f-6a950c0aac84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:08.524947 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82f729f-a885-42c6-b42f-6a950c0aac84 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:08.534537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 541/1076] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:10:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:08.540168 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e25ba-ea14-44b1-98a7-bfce0db5aa5a None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:08.650479 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e25ba-ea14-44b1-98a7-bfce0db5aa5a None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:08.674501 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:08.686419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c37a31-65e7-4d83-874b-41766db3ac06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:08.687097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c37a31-65e7-4d83-874b-41766db3ac06 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:08.687376 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c37a31-65e7-4d83-874b-41766db3ac06 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:08.687648 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c37a31-65e7-4d83-874b-41766db3ac06 None admin] PATH_INFO: `/v3/users/3281369f31b54b32804554ba9f67673b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:08.689079 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c37a31-65e7-4d83-874b-41766db3ac06 None admin] RBAC: Authorizing `identity:get_user(user_id=3281369f31b54b32804554ba9f67673b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:08.690555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c37a31-65e7-4d83-874b-41766db3ac06 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:08.692315 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 536/1077] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:10:08 2022] GET /identity/v3/users/3281369f31b54b32804554ba9f67673b => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:08.697914 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f729f-a885-42c6-b42f-6a950c0aac84 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:08.719941 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:08.805155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f729f-a885-42c6-b42f-6a950c0aac84 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:08.839770 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc2b4d2-22f4-4cf7-8604-70a55d20a9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:08.840420 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2b4d2-22f4-4cf7-8604-70a55d20a9c3 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:08.840710 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2b4d2-22f4-4cf7-8604-70a55d20a9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:08.840989 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2b4d2-22f4-4cf7-8604-70a55d20a9c3 None admin] PATH_INFO: `/v3/projects/b4e2148743504b999b0e30a1fb3e97c7` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:08.842181 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc2b4d2-22f4-4cf7-8604-70a55d20a9c3 None admin] RBAC: Authorizing `identity:get_project(project_id=b4e2148743504b999b0e30a1fb3e97c7)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:08.844257 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc2b4d2-22f4-4cf7-8604-70a55d20a9c3 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:08.850825 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 542/1078] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:10:08 2022] GET /identity/v3/projects/b4e2148743504b999b0e30a1fb3e97c7 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:10:08.852286 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c37a31-65e7-4d83-874b-41766db3ac06 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:08.954531 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c37a31-65e7-4d83-874b-41766db3ac06 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:08.988354 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6fb343-1168-46df-b0b0-054282956626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:08.989030 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fb343-1168-46df-b0b0-054282956626 None admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:08.989311 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fb343-1168-46df-b0b0-054282956626 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:08.989584 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fb343-1168-46df-b0b0-054282956626 None admin] PATH_INFO: `/v3/projects/b4e2148743504b999b0e30a1fb3e97c7/users/3281369f31b54b32804554ba9f67673b/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:08.991525 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6fb343-1168-46df-b0b0-054282956626 None admin] RBAC: Authorizing `identity:create_grant(project_id=b4e2148743504b999b0e30a1fb3e97c7, user_id=3281369f31b54b32804554ba9f67673b, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:08.992949 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6fb343-1168-46df-b0b0-054282956626 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:09.006966 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 537/1079] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 27 04:10:08 2022] PUT /identity/v3/projects/b4e2148743504b999b0e30a1fb3e97c7/users/3281369f31b54b32804554ba9f67673b/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:10:09.509379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ee700-cf94-4c52-a045-cf3df6a28c6d None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:09.509888 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ee700-cf94-4c52-a045-cf3df6a28c6d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:09.510019 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ee700-cf94-4c52-a045-cf3df6a28c6d None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:09.511348 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 543/1080] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:10:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:10:09.517136 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24277cdf-f5f6-45df-a8fa-7046702b9eab None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:09.517430 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24277cdf-f5f6-45df-a8fa-7046702b9eab None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:09.517710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24277cdf-f5f6-45df-a8fa-7046702b9eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:09.536351 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-24277cdf-f5f6-45df-a8fa-7046702b9eab None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:10:09.537531 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-24277cdf-f5f6-45df-a8fa-7046702b9eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:10:09.585158 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:09.607165 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 538/1081] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:10:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:10:09.628925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ee700-cf94-4c52-a045-cf3df6a28c6d None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:09.717806 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:09.736248 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ee700-cf94-4c52-a045-cf3df6a28c6d None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:09.770686 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b2a0d4-b937-4055-880b-5086256fec0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:09.771316 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2a0d4-b937-4055-880b-5086256fec0e None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:09.771597 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2a0d4-b937-4055-880b-5086256fec0e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:09.771871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2a0d4-b937-4055-880b-5086256fec0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:09.840339 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b2a0d4-b937-4055-880b-5086256fec0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:09.841787 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b2a0d4-b937-4055-880b-5086256fec0e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:09.927277 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:09.948686 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 544/1082] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:10:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 04:10:09.950373 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24277cdf-f5f6-45df-a8fa-7046702b9eab None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:10.050201 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24277cdf-f5f6-45df-a8fa-7046702b9eab None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:10.088004 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a7b71e-9bda-42c4-b629-3bfe2ec14b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:10.088682 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7b71e-9bda-42c4-b629-3bfe2ec14b38 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:10.088963 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7b71e-9bda-42c4-b629-3bfe2ec14b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:10.089234 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7b71e-9bda-42c4-b629-3bfe2ec14b38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:10.101801 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a7b71e-9bda-42c4-b629-3bfe2ec14b38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:10.103312 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a7b71e-9bda-42c4-b629-3bfe2ec14b38 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:10.118815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-84a7b71e-9bda-42c4-b629-3bfe2ec14b38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:10:10.118815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:10:10.118815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:10:10.118815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:10:10.118815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:10:10.118815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:10:10.118815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:10:10.118815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:10:10.118815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:10:10.118815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:10:10.118815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:10:10.118815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:10:10.118815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:10:10.118815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:10:10.118815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:10:10.119759 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:10:10.119759 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:10:10.119759 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:10:10.119759 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:10:10.119759 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:10:10.119759 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:10:10.119759 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:10:10.119759 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:10:10.119759 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:10:10.119759 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:10:10.119759 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:10:10.119759 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:10:10.119759 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:10:10.119759 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:10:10.119759 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:10:10.119759 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:10:10.119759 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:10:10.120840 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:10:10.120840 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:10:10.120840 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:10:10.120840 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:10:10.120840 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 539/1083] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:10:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:10:10.125955 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b2a0d4-b937-4055-880b-5086256fec0e None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:10.225137 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b2a0d4-b937-4055-880b-5086256fec0e None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:10.259464 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a1d48-a684-4aa5-8262-7a6febcfed0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:10.260091 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a1d48-a684-4aa5-8262-7a6febcfed0d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:10.260374 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a1d48-a684-4aa5-8262-7a6febcfed0d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:10.260646 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a1d48-a684-4aa5-8262-7a6febcfed0d None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:10.261341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a1d48-a684-4aa5-8262-7a6febcfed0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:10.261644 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a1d48-a684-4aa5-8262-7a6febcfed0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:10.263038 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a1d48-a684-4aa5-8262-7a6febcfed0d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:10.265132 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca867ac-2635-45be-affa-4562523741cf None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:10.265500 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca867ac-2635-45be-affa-4562523741cf None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:10.265786 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca867ac-2635-45be-affa-4562523741cf None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:10.267177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 540/1084] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:10:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:10:10.272125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636c382-1da3-4bd2-9a99-1190bad0f440 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:10.272411 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636c382-1da3-4bd2-9a99-1190bad0f440 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:10.272686 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636c382-1da3-4bd2-9a99-1190bad0f440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:10.275319 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 545/1085] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:10:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:10.299266 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-a636c382-1da3-4bd2-9a99-1190bad0f440 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:10:10.300554 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-a636c382-1da3-4bd2-9a99-1190bad0f440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:10:10.333272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:10.379377 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 541/1086] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:10:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 04:10:10.402611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a1d48-a684-4aa5-8262-7a6febcfed0d None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:10.454831 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:10.513593 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a1d48-a684-4aa5-8262-7a6febcfed0d None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:10.552256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade7d1e2-87c8-4f93-b6a8-c7cb49f0d0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:10.552828 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7d1e2-87c8-4f93-b6a8-c7cb49f0d0d4 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:10.553106 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7d1e2-87c8-4f93-b6a8-c7cb49f0d0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:10.553379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7d1e2-87c8-4f93-b6a8-c7cb49f0d0d4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:10.564761 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade7d1e2-87c8-4f93-b6a8-c7cb49f0d0d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:10:10.570236 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-ade7d1e2-87c8-4f93-b6a8-c7cb49f0d0d4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 04:10:10.570236 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:10:10.570236 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:10:10.570236 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:10:10.570236 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:10:10.570236 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:10:10.570236 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:10:10.570236 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:10:10.570236 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:10:10.570236 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:10:10.570236 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:10:10.570236 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:10:10.570236 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:10:10.570236 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:10:10.570236 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:10:10.570236 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:10:10.571185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:10:10.571185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:10:10.571185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:10:10.571185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:10:10.571185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:10:10.571185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:10:10.571185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:10:10.571185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:10:10.571185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:10:10.571185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:10:10.571185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:10:10.571185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:10:10.571185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:10:10.571185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:10:10.571185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:10:10.571185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:10:10.572040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:10:10.572040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:10:10.572040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:10:10.572040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:10:10.572040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:10:10.572040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:10:10.572040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:10:10.572040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:10:10.572040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:10:10.572040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:10:10.572040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 04:10:10.572040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:10:10.572040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 546/1087] 10.222.0.46 () {56 vars in 1120 bytes} [Wed Jul 27 04:10:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:10:10.577448 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a636c382-1da3-4bd2-9a99-1190bad0f440 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:10.677405 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a636c382-1da3-4bd2-9a99-1190bad0f440 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:10.711869 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206226f-9109-46a9-83ed-e25beff279f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:10.712486 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206226f-9109-46a9-83ed-e25beff279f3 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:10.712763 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206226f-9109-46a9-83ed-e25beff279f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:10.713032 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206226f-9109-46a9-83ed-e25beff279f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:10.713749 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4206226f-9109-46a9-83ed-e25beff279f3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:10.714076 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4206226f-9109-46a9-83ed-e25beff279f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:10.715431 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4206226f-9109-46a9-83ed-e25beff279f3 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:10.727813 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 542/1088] 10.222.0.46 () {56 vars in 1115 bytes} [Wed Jul 27 04:10:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:10:10.729198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade7d1e2-87c8-4f93-b6a8-c7cb49f0d0d4 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:10.827187 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade7d1e2-87c8-4f93-b6a8-c7cb49f0d0d4 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:10.833798 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:10.864500 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d10c463-f7d2-4f46-977d-49d97819d12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:10.869995 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10c463-f7d2-4f46-977d-49d97819d12f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:10.869995 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10c463-f7d2-4f46-977d-49d97819d12f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:10.869995 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10c463-f7d2-4f46-977d-49d97819d12f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:10.877820 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d10c463-f7d2-4f46-977d-49d97819d12f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:10.879360 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d10c463-f7d2-4f46-977d-49d97819d12f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:10.893434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-8d10c463-f7d2-4f46-977d-49d97819d12f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:10:10.893434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:10:10.893434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:10:10.893434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:10:10.893434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:10:10.893434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:10:10.893434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:10:10.893434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:10:10.893434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:10:10.893434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:10:10.893434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:10:10.893434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:10:10.893434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:10:10.893434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:10:10.893434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:10:10.894545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:10:10.894545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:10:10.894545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:10:10.894545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:10:10.894545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:10:10.894545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:10:10.894545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:10:10.894545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:10:10.894545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:10:10.894545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:10:10.894545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:10:10.894545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:10:10.894545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:10:10.894545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:10:10.894545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:10:10.894545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:10:10.894545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:10:10.895461 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:10:10.895461 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:10:10.895461 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:10:10.895461 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:10:10.895461 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 547/1089] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:10:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:10:10.900376 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206226f-9109-46a9-83ed-e25beff279f3 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:10.929668 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:11.005364 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206226f-9109-46a9-83ed-e25beff279f3 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:11.018258 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fcd2c6-2f56-4318-b47f-6f23680a1c36 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:11.018548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fcd2c6-2f56-4318-b47f-6f23680a1c36 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:11.018821 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fcd2c6-2f56-4318-b47f-6f23680a1c36 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:11.020183 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 548/1090] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:10:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:10:11.025509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99185c22-2b3c-4186-8ccc-660729ef95d5 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:11.025792 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99185c22-2b3c-4186-8ccc-660729ef95d5 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:11.026096 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99185c22-2b3c-4186-8ccc-660729ef95d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:11.040481 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb4684-60df-4036-8294-55a2693c3062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:11.041107 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb4684-60df-4036-8294-55a2693c3062 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:11.041392 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb4684-60df-4036-8294-55a2693c3062 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:11.041667 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb4684-60df-4036-8294-55a2693c3062 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:11.042397 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb4684-60df-4036-8294-55a2693c3062 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:11.042700 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb4684-60df-4036-8294-55a2693c3062 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:11.044060 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb4684-60df-4036-8294-55a2693c3062 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:11.045652 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-99185c22-2b3c-4186-8ccc-660729ef95d5 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:10:11.047629 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-99185c22-2b3c-4186-8ccc-660729ef95d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:10:11.057599 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 543/1091] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:10:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:11.063683 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb4684-60df-4036-8294-55a2693c3062 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:11.122224 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 549/1092] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:10:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 04:10:11.145520 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99185c22-2b3c-4186-8ccc-660729ef95d5 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:11.160557 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:11.175968 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb4684-60df-4036-8294-55a2693c3062 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:11.211021 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fff919-883d-4784-804a-825461614bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:11.211660 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fff919-883d-4784-804a-825461614bd1 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:11.211941 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fff919-883d-4784-804a-825461614bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:11.212210 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fff919-883d-4784-804a-825461614bd1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:11.219347 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fff919-883d-4784-804a-825461614bd1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:11.220806 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fff919-883d-4784-804a-825461614bd1 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:11.230856 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:11.232541 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-b4fff919-883d-4784-804a-825461614bd1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 04:10:11.232541 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:10:11.232541 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:10:11.232541 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:10:11.232541 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:10:11.232541 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:10:11.232541 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:10:11.232541 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:10:11.232541 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:10:11.232541 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:10:11.232541 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:10:11.232541 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:10:11.232541 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:10:11.232541 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:10:11.232541 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:10:11.232541 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:10:11.233559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:10:11.233559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:10:11.233559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:10:11.233559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:10:11.233559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:10:11.233559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:10:11.233559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:10:11.233559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:10:11.233559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:10:11.233559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:10:11.233559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:10:11.233559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:10:11.233559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:10:11.233559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:10:11.233559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:10:11.233559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:10:11.233559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:10:11.233559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:10:11.235234 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:10:11.235234 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:10:11.235234 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:10:11.235234 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:10:11.235234 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:10:11.235234 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:10:11.235234 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:10:11.235234 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:10:11.235234 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 04:10:11.235234 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:10:11.235234 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 544/1093] 10.222.0.46 () {56 vars in 1093 bytes} [Wed Jul 27 04:10:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:10:11.239399 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fff919-883d-4784-804a-825461614bd1 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:11.245568 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99185c22-2b3c-4186-8ccc-660729ef95d5 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:11.280394 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19eac8-d3f9-4b22-8797-a0eaf49a91e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:11.281065 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19eac8-d3f9-4b22-8797-a0eaf49a91e4 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:11.281519 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19eac8-d3f9-4b22-8797-a0eaf49a91e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:11.281798 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19eac8-d3f9-4b22-8797-a0eaf49a91e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:11.291718 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa19eac8-d3f9-4b22-8797-a0eaf49a91e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:10:11.296672 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-aa19eac8-d3f9-4b22-8797-a0eaf49a91e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:10:11.296672 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:10:11.296672 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:10:11.296672 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:10:11.296672 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:10:11.296672 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:10:11.296672 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:10:11.296672 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:10:11.296672 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:10:11.296672 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:10:11.296672 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:10:11.296672 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:10:11.296672 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:10:11.296672 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:10:11.296672 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:10:11.296672 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:10:11.297653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:10:11.297653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:10:11.297653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:10:11.297653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:10:11.297653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:10:11.297653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:10:11.297653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:10:11.297653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:10:11.297653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:10:11.297653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:10:11.297653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:10:11.297653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:10:11.297653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:10:11.297653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:10:11.297653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:10:11.297653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:10:11.298903 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:10:11.298903 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:10:11.298903 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:10:11.298903 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:10:11.298903 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:10:11.298903 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:10:11.298903 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:10:11.298903 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:10:11.298903 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:10:11.298903 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:10:11.298903 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:10:11.298903 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:10:11.298903 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 550/1094] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:10:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:10:11.303726 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19eac8-d3f9-4b22-8797-a0eaf49a91e4 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:11.337304 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fff919-883d-4784-804a-825461614bd1 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:11.370328 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78425f79-bb48-461e-a63f-df6cc10b54fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:11.370935 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78425f79-bb48-461e-a63f-df6cc10b54fa None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:11.371217 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78425f79-bb48-461e-a63f-df6cc10b54fa None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:11.371504 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78425f79-bb48-461e-a63f-df6cc10b54fa None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:11.372342 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78425f79-bb48-461e-a63f-df6cc10b54fa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:11.372600 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78425f79-bb48-461e-a63f-df6cc10b54fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:11.374087 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78425f79-bb48-461e-a63f-df6cc10b54fa None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:11.392036 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 545/1095] 10.222.0.46 () {56 vars in 1133 bytes} [Wed Jul 27 04:10:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:10:11.397607 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78425f79-bb48-461e-a63f-df6cc10b54fa None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:11.400514 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19eac8-d3f9-4b22-8797-a0eaf49a91e4 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:11.401002 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:11.434603 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09259ccf-ff3f-4ecb-abf4-3a9b1b387d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:11.435222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09259ccf-ff3f-4ecb-abf4-3a9b1b387d67 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:11.435508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09259ccf-ff3f-4ecb-abf4-3a9b1b387d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:11.435787 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09259ccf-ff3f-4ecb-abf4-3a9b1b387d67 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:11.436522 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09259ccf-ff3f-4ecb-abf4-3a9b1b387d67 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:11.437395 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09259ccf-ff3f-4ecb-abf4-3a9b1b387d67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:11.439052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09259ccf-ff3f-4ecb-abf4-3a9b1b387d67 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:11.447209 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 551/1096] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:10:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:10:11.452990 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09259ccf-ff3f-4ecb-abf4-3a9b1b387d67 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:11.467609 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:11.499492 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78425f79-bb48-461e-a63f-df6cc10b54fa None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:11.533967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2b02b-7c0a-4010-8541-12a9bde0b80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:11.534630 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2b02b-7c0a-4010-8541-12a9bde0b80f None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:11.534913 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2b02b-7c0a-4010-8541-12a9bde0b80f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:11.535187 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2b02b-7c0a-4010-8541-12a9bde0b80f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:11.555546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09259ccf-ff3f-4ecb-abf4-3a9b1b387d67 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:11.588964 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d847685-55c3-43bf-9229-4f448372f627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:11.589597 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d847685-55c3-43bf-9229-4f448372f627 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:11.589873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d847685-55c3-43bf-9229-4f448372f627 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:11.590176 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d847685-55c3-43bf-9229-4f448372f627 None admin] PATH_INFO: `/v3/users/3281369f31b54b32804554ba9f67673b` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:11.591329 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d847685-55c3-43bf-9229-4f448372f627 None admin] RBAC: Authorizing `identity:get_user(user_id=3281369f31b54b32804554ba9f67673b)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:11.592783 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d847685-55c3-43bf-9229-4f448372f627 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:11.594629 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 552/1097] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:10:11 2022] GET /identity/v3/users/3281369f31b54b32804554ba9f67673b => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:11.600220 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d847685-55c3-43bf-9229-4f448372f627 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:11.602098 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac2b02b-7c0a-4010-8541-12a9bde0b80f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:11.603551 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac2b02b-7c0a-4010-8541-12a9bde0b80f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:11.606173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:11.678713 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:11.701346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d847685-55c3-43bf-9229-4f448372f627 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:11.706209 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 546/1098] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:10:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 27 04:10:11.707210 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2b02b-7c0a-4010-8541-12a9bde0b80f None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:11.735347 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb5a0a-d877-4b33-b4b4-e7c093cf8a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:11.735966 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb5a0a-d877-4b33-b4b4-e7c093cf8a4a None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:11.736241 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb5a0a-d877-4b33-b4b4-e7c093cf8a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:11.736514 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb5a0a-d877-4b33-b4b4-e7c093cf8a4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:11.802727 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb5a0a-d877-4b33-b4b4-e7c093cf8a4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:11.804179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb5a0a-d877-4b33-b4b4-e7c093cf8a4a None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:11.805433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2b02b-7c0a-4010-8541-12a9bde0b80f None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:11.808107 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:11.840747 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918cc3c-fd37-44ff-a0a7-5763342e3e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:11.841391 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918cc3c-fd37-44ff-a0a7-5763342e3e29 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:11.841668 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918cc3c-fd37-44ff-a0a7-5763342e3e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:11.841941 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918cc3c-fd37-44ff-a0a7-5763342e3e29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:11.852016 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3918cc3c-fd37-44ff-a0a7-5763342e3e29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:11.853474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3918cc3c-fd37-44ff-a0a7-5763342e3e29 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:11.868953 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-3918cc3c-fd37-44ff-a0a7-5763342e3e29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:10:11.868953 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:10:11.868953 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:10:11.868953 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:10:11.868953 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:10:11.868953 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:10:11.868953 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:10:11.868953 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:10:11.868953 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:10:11.868953 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:10:11.868953 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:10:11.868953 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:10:11.868953 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:10:11.868953 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:10:11.868953 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:10:11.869930 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:10:11.869930 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:10:11.869930 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:10:11.869930 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:10:11.869930 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:10:11.869930 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:10:11.869930 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:10:11.869930 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:10:11.869930 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:10:11.869930 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:10:11.869930 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:10:11.869930 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:10:11.869930 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:10:11.869930 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:10:11.869930 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:10:11.869930 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:10:11.869930 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:10:11.870876 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:10:11.870876 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:10:11.870876 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:10:11.870876 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:10:11.870876 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 547/1099] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:10:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:10:11.876159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918cc3c-fd37-44ff-a0a7-5763342e3e29 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:11.882973 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:11.905439 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 553/1100] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:10:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 27 04:10:11.912070 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb5a0a-d877-4b33-b4b4-e7c093cf8a4a None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:11.977593 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918cc3c-fd37-44ff-a0a7-5763342e3e29 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:12.009691 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb5a0a-d877-4b33-b4b4-e7c093cf8a4a None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:12.011593 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b662ae-6ea6-442e-84c9-fd905a79b45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:12.012204 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b662ae-6ea6-442e-84c9-fd905a79b45e None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:12.012484 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b662ae-6ea6-442e-84c9-fd905a79b45e None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:12.012764 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b662ae-6ea6-442e-84c9-fd905a79b45e None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:12.013457 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b662ae-6ea6-442e-84c9-fd905a79b45e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:12.013758 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b662ae-6ea6-442e-84c9-fd905a79b45e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:12.015869 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b662ae-6ea6-442e-84c9-fd905a79b45e None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:12.027502 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 548/1101] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:10:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:10:12.033289 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b662ae-6ea6-442e-84c9-fd905a79b45e None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:12.044710 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e188a-c99d-4694-b8ee-d1d33baf3d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:12.045358 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e188a-c99d-4694-b8ee-d1d33baf3d32 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:12.045640 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e188a-c99d-4694-b8ee-d1d33baf3d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:12.045921 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e188a-c99d-4694-b8ee-d1d33baf3d32 None admin] PATH_INFO: `/v3/projects/b4e2148743504b999b0e30a1fb3e97c7` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:12.047276 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e188a-c99d-4694-b8ee-d1d33baf3d32 None admin] RBAC: Authorizing `identity:get_project(project_id=b4e2148743504b999b0e30a1fb3e97c7)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:12.048704 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e188a-c99d-4694-b8ee-d1d33baf3d32 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:12.051634 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 554/1102] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:10:11 2022] GET /identity/v3/projects/b4e2148743504b999b0e30a1fb3e97c7 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:10:12.051770 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:12.056381 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e188a-c99d-4694-b8ee-d1d33baf3d32 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:12.103141 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:12.134685 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b662ae-6ea6-442e-84c9-fd905a79b45e None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:12.156935 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e188a-c99d-4694-b8ee-d1d33baf3d32 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:12.168347 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3f59c-8bfc-4de5-8975-5718ec6eaa84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:12.168983 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3f59c-8bfc-4de5-8975-5718ec6eaa84 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:12.169263 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3f59c-8bfc-4de5-8975-5718ec6eaa84 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:12.169540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3f59c-8bfc-4de5-8975-5718ec6eaa84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:12.176120 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3f59c-8bfc-4de5-8975-5718ec6eaa84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:12.177553 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3f59c-8bfc-4de5-8975-5718ec6eaa84 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:12.190172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-c9b3f59c-8bfc-4de5-8975-5718ec6eaa84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:10:12.190172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:10:12.190172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:10:12.190172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:10:12.190172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:10:12.190172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:10:12.190172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:10:12.190172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:10:12.190172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:10:12.190172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:10:12.190172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:10:12.190172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:10:12.190172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:10:12.190172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:10:12.190172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:10:12.191254 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:10:12.191254 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:10:12.191254 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:10:12.191254 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:10:12.191254 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:10:12.191254 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:10:12.191254 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:10:12.191254 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:10:12.191254 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:10:12.191254 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:10:12.191254 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:10:12.191254 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:10:12.191254 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:10:12.191254 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:10:12.191254 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:10:12.191254 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:10:12.191254 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:10:12.192631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:10:12.192631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:10:12.192631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:10:12.192631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:10:12.192631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:10:12.192631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:10:12.192631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:10:12.192631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:10:12.192631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 549/1103] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:10:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:10:12.193309 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ab6ac0-bf13-47c2-94f0-51809267648c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:12.193946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab6ac0-bf13-47c2-94f0-51809267648c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:12.194329 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab6ac0-bf13-47c2-94f0-51809267648c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:12.194663 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab6ac0-bf13-47c2-94f0-51809267648c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:12.195552 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ab6ac0-bf13-47c2-94f0-51809267648c None admin] RBAC: Adding query filter params (role.id=cf771a8e94ee46149eb081347fea6d67, scope.project.id=b4e2148743504b999b0e30a1fb3e97c7, user.id=3281369f31b54b32804554ba9f67673b) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:12.195945 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ab6ac0-bf13-47c2-94f0-51809267648c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:12.197048 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3f59c-8bfc-4de5-8975-5718ec6eaa84 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:12.197595 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ab6ac0-bf13-47c2-94f0-51809267648c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:12.210070 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 555/1104] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:10:12 2022] GET /identity/v3/role_assignments?user.id=3281369f31b54b32804554ba9f67673b&scope.project.id=b4e2148743504b999b0e30a1fb3e97c7&role.id=cf771a8e94ee46149eb081347fea6d67 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:10:12.279735 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:12.304856 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3f59c-8bfc-4de5-8975-5718ec6eaa84 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:12.340639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b0c23-7d9e-45a6-8ecb-c7af44ba09fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:12.341275 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b0c23-7d9e-45a6-8ecb-c7af44ba09fd None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:12.341558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b0c23-7d9e-45a6-8ecb-c7af44ba09fd None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:12.341837 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b0c23-7d9e-45a6-8ecb-c7af44ba09fd None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:12.342636 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b0c23-7d9e-45a6-8ecb-c7af44ba09fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:12.342945 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b0c23-7d9e-45a6-8ecb-c7af44ba09fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:12.345368 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b0c23-7d9e-45a6-8ecb-c7af44ba09fd None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:12.358201 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 550/1105] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:10:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:10:12.364183 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ab6ac0-bf13-47c2-94f0-51809267648c None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:12.466552 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ab6ac0-bf13-47c2-94f0-51809267648c None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:12.477078 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:12.501362 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c788a32-a561-46ae-b32b-9f792d5cbadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:12.502007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c788a32-a561-46ae-b32b-9f792d5cbadf None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:12.502299 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c788a32-a561-46ae-b32b-9f792d5cbadf None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:12.502578 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c788a32-a561-46ae-b32b-9f792d5cbadf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:12.503433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c788a32-a561-46ae-b32b-9f792d5cbadf None admin] RBAC: Adding query filter params (role.id=3298581080b940c1ab0a41e5bbc8953f, scope.project.id=e646d320aa5a4f78bfccaf941cf46478, user.id=9dc9ea8f89c14a2a8e3272f750647dfd) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:12.503752 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c788a32-a561-46ae-b32b-9f792d5cbadf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:12.505185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c788a32-a561-46ae-b32b-9f792d5cbadf None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:12.510923 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 556/1106] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:10:12 2022] GET /identity/v3/role_assignments?user.id=9dc9ea8f89c14a2a8e3272f750647dfd&scope.project.id=e646d320aa5a4f78bfccaf941cf46478&role.id=3298581080b940c1ab0a41e5bbc8953f => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:14.221041 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344cf49c-3f8e-4d30-9d2b-cb3b8cedbb2a None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:14.221567 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344cf49c-3f8e-4d30-9d2b-cb3b8cedbb2a None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:14.221662 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344cf49c-3f8e-4d30-9d2b-cb3b8cedbb2a None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:14.222983 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 551/1107] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:10:14.228112 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e7ec0-ebec-4cf4-b661-88f2c3a1d287 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:14.228504 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e7ec0-ebec-4cf4-b661-88f2c3a1d287 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:14.228662 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e7ec0-ebec-4cf4-b661-88f2c3a1d287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:14.247007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-be3e7ec0-ebec-4cf4-b661-88f2c3a1d287 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:10:14.248190 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-be3e7ec0-ebec-4cf4-b661-88f2c3a1d287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:10:14.311357 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 557/1108] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:10:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:10:14.335870 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344cf49c-3f8e-4d30-9d2b-cb3b8cedbb2a None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:14.433306 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344cf49c-3f8e-4d30-9d2b-cb3b8cedbb2a None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:14.470157 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42767c7-5db7-46d5-a0bd-b96542ae0b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:14.470805 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42767c7-5db7-46d5-a0bd-b96542ae0b14 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:14.471136 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42767c7-5db7-46d5-a0bd-b96542ae0b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:14.471356 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42767c7-5db7-46d5-a0bd-b96542ae0b14 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:14.472496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42767c7-5db7-46d5-a0bd-b96542ae0b14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:14.473941 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42767c7-5db7-46d5-a0bd-b96542ae0b14 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:14.475547 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 552/1109] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:10:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:14.481297 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e7ec0-ebec-4cf4-b661-88f2c3a1d287 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:14.576608 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e7ec0-ebec-4cf4-b661-88f2c3a1d287 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:14.579024 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:14.612273 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1488a530-b1df-4c23-95b3-4cc4c62b553c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:14.612889 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488a530-b1df-4c23-95b3-4cc4c62b553c None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:14.613172 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488a530-b1df-4c23-95b3-4cc4c62b553c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:14.613449 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488a530-b1df-4c23-95b3-4cc4c62b553c None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:14.614176 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1488a530-b1df-4c23-95b3-4cc4c62b553c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:14.615598 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1488a530-b1df-4c23-95b3-4cc4c62b553c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:14.636648 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 558/1110] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 27 04:10:14 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:10:16.701052 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50662c76-a008-45b2-b2aa-4147a5444e20 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:16.701597 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50662c76-a008-45b2-b2aa-4147a5444e20 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:16.701597 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50662c76-a008-45b2-b2aa-4147a5444e20 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:16.703038 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 553/1111] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:10:16.708699 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d2809-a17f-4d23-9c95-5ce439dfc6eb None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:16.709065 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d2809-a17f-4d23-9c95-5ce439dfc6eb None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:16.709206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d2809-a17f-4d23-9c95-5ce439dfc6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:16.727075 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-4b1d2809-a17f-4d23-9c95-5ce439dfc6eb None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:10:16.728260 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-4b1d2809-a17f-4d23-9c95-5ce439dfc6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:10:16.792202 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 559/1112] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:10:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:10:16.818500 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50662c76-a008-45b2-b2aa-4147a5444e20 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:16.822280 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:16.918364 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50662c76-a008-45b2-b2aa-4147a5444e20 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:16.951827 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e654ba-3736-4950-ac7b-eee982593979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:16.953014 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e654ba-3736-4950-ac7b-eee982593979 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:16.953516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e654ba-3736-4950-ac7b-eee982593979 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:16.953794 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e654ba-3736-4950-ac7b-eee982593979 None admin] PATH_INFO: `/v3/roles/cdfc87279ea24b65bae3961035133725` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:16.955260 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e654ba-3736-4950-ac7b-eee982593979 None admin] RBAC: Authorizing `identity:get_role(role_id=cdfc87279ea24b65bae3961035133725)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:16.956633 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e654ba-3736-4950-ac7b-eee982593979 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:16.957947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 554/1113] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:10:16 2022] GET /identity/v3/roles/cdfc87279ea24b65bae3961035133725 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:16.963769 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1d2809-a17f-4d23-9c95-5ce439dfc6eb None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:17.050844 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:17.061205 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1d2809-a17f-4d23-9c95-5ce439dfc6eb None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:17.095090 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f582c1f-0418-4170-afc3-7785d1b3a75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:17.095729 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f582c1f-0418-4170-afc3-7785d1b3a75d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:17.096015 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f582c1f-0418-4170-afc3-7785d1b3a75d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:17.096293 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f582c1f-0418-4170-afc3-7785d1b3a75d None admin] PATH_INFO: `/v3/users/9c2db95bdf92491b81a58119944240a5` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:17.109293 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f582c1f-0418-4170-afc3-7785d1b3a75d None admin] RBAC: Authorizing `identity:get_user(user_id=9c2db95bdf92491b81a58119944240a5)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:17.110882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f582c1f-0418-4170-afc3-7785d1b3a75d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:17.112500 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 560/1114] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:10:16 2022] GET /identity/v3/users/9c2db95bdf92491b81a58119944240a5 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:17.118206 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e654ba-3736-4950-ac7b-eee982593979 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:17.186169 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:17.214625 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e654ba-3736-4950-ac7b-eee982593979 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:17.247751 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112ae9c-a586-4059-9787-36e167687c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:17.248354 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112ae9c-a586-4059-9787-36e167687c44 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:17.248631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112ae9c-a586-4059-9787-36e167687c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:17.248899 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112ae9c-a586-4059-9787-36e167687c44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:17.315109 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5112ae9c-a586-4059-9787-36e167687c44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:17.316551 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5112ae9c-a586-4059-9787-36e167687c44 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:17.383145 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:17.415149 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 555/1115] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:10:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 04:10:17.421028 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f582c1f-0418-4170-afc3-7785d1b3a75d None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:17.518378 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f582c1f-0418-4170-afc3-7785d1b3a75d None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:17.551078 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2391ef8a-f0b8-4f54-acc4-d77572d24628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:17.551717 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2391ef8a-f0b8-4f54-acc4-d77572d24628 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:17.551999 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2391ef8a-f0b8-4f54-acc4-d77572d24628 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:17.552277 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2391ef8a-f0b8-4f54-acc4-d77572d24628 None admin] PATH_INFO: `/v3/projects/b4e2148743504b999b0e30a1fb3e97c7` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:17.553431 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2391ef8a-f0b8-4f54-acc4-d77572d24628 None admin] RBAC: Authorizing `identity:get_project(project_id=b4e2148743504b999b0e30a1fb3e97c7)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:17.554885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2391ef8a-f0b8-4f54-acc4-d77572d24628 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:17.556540 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 561/1116] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:10:17 2022] GET /identity/v3/projects/b4e2148743504b999b0e30a1fb3e97c7 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:17.563261 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112ae9c-a586-4059-9787-36e167687c44 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:17.658437 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112ae9c-a586-4059-9787-36e167687c44 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:17.691372 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c7a49-50a4-48bd-8437-8bed17eb3c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:17.691977 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c7a49-50a4-48bd-8437-8bed17eb3c58 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:17.692261 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c7a49-50a4-48bd-8437-8bed17eb3c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:17.692537 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c7a49-50a4-48bd-8437-8bed17eb3c58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:17.693324 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7c7a49-50a4-48bd-8437-8bed17eb3c58 None admin] RBAC: Adding query filter params (role.id=cdfc87279ea24b65bae3961035133725, scope.project.id=b4e2148743504b999b0e30a1fb3e97c7, user.id=9c2db95bdf92491b81a58119944240a5) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:17.693634 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7c7a49-50a4-48bd-8437-8bed17eb3c58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:17.695089 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7c7a49-50a4-48bd-8437-8bed17eb3c58 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:17.700341 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 556/1117] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:10:17 2022] GET /identity/v3/role_assignments?user.id=9c2db95bdf92491b81a58119944240a5&scope.project.id=b4e2148743504b999b0e30a1fb3e97c7&role.id=cdfc87279ea24b65bae3961035133725 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:19.671966 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42091a70-928a-4073-8cda-0bef28ce7960 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:19.672551 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42091a70-928a-4073-8cda-0bef28ce7960 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:19.672551 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42091a70-928a-4073-8cda-0bef28ce7960 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:19.674063 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 562/1118] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:10:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:10:19.679185 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64daff-3f98-4333-a183-13f1e31a4f2e None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:19.679612 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64daff-3f98-4333-a183-13f1e31a4f2e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:19.679747 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64daff-3f98-4333-a183-13f1e31a4f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:19.698225 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-1e64daff-3f98-4333-a183-13f1e31a4f2e None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:10:19.699396 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-1e64daff-3f98-4333-a183-13f1e31a4f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:10:19.727719 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:19.765480 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 557/1119] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:10:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:10:19.787452 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42091a70-928a-4073-8cda-0bef28ce7960 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:19.805309 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:19.886409 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42091a70-928a-4073-8cda-0bef28ce7960 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:19.918208 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30047565-769a-45b4-9a37-2cdd53bc33e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:19.918835 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30047565-769a-45b4-9a37-2cdd53bc33e4 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:19.919113 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30047565-769a-45b4-9a37-2cdd53bc33e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:19.919384 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30047565-769a-45b4-9a37-2cdd53bc33e4 None admin] PATH_INFO: `/v3/roles/cdfc87279ea24b65bae3961035133725` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:19.920802 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30047565-769a-45b4-9a37-2cdd53bc33e4 None admin] RBAC: Authorizing `identity:get_role(role_id=cdfc87279ea24b65bae3961035133725)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:19.922719 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30047565-769a-45b4-9a37-2cdd53bc33e4 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:19.924031 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 563/1120] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:10:19 2022] GET /identity/v3/roles/cdfc87279ea24b65bae3961035133725 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:19.929958 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64daff-3f98-4333-a183-13f1e31a4f2e None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:20.030920 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64daff-3f98-4333-a183-13f1e31a4f2e None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:20.066598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fac919-e1a6-465b-a1ba-655048c65046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:20.067232 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fac919-e1a6-465b-a1ba-655048c65046 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:20.067513 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fac919-e1a6-465b-a1ba-655048c65046 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:20.067787 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fac919-e1a6-465b-a1ba-655048c65046 None admin] PATH_INFO: `/v3/users/9c2db95bdf92491b81a58119944240a5` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:20.068937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fac919-e1a6-465b-a1ba-655048c65046 None admin] RBAC: Authorizing `identity:get_user(user_id=9c2db95bdf92491b81a58119944240a5)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:20.070406 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fac919-e1a6-465b-a1ba-655048c65046 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:20.072003 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 558/1121] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:10:19 2022] GET /identity/v3/users/9c2db95bdf92491b81a58119944240a5 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:20.078463 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30047565-769a-45b4-9a37-2cdd53bc33e4 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:20.158463 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:20.173863 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30047565-769a-45b4-9a37-2cdd53bc33e4 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:20.207011 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcbef08-81c4-4e25-b57d-2ffb6d6a0cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:20.207633 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcbef08-81c4-4e25-b57d-2ffb6d6a0cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:20.207911 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcbef08-81c4-4e25-b57d-2ffb6d6a0cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:20.208183 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcbef08-81c4-4e25-b57d-2ffb6d6a0cf1 None admin] PATH_INFO: `/v3/projects/b4e2148743504b999b0e30a1fb3e97c7` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:20.209325 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcbef08-81c4-4e25-b57d-2ffb6d6a0cf1 None admin] RBAC: Authorizing `identity:get_project(project_id=b4e2148743504b999b0e30a1fb3e97c7)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:20.210767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcbef08-81c4-4e25-b57d-2ffb6d6a0cf1 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:20.212396 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 564/1122] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:10:20 2022] GET /identity/v3/projects/b4e2148743504b999b0e30a1fb3e97c7 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:20.218611 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fac919-e1a6-465b-a1ba-655048c65046 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:20.251901 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:20.328816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fac919-e1a6-465b-a1ba-655048c65046 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:20.363010 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56204ef8-7e1e-4061-9ad7-fb5bbdaceb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:20.363681 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56204ef8-7e1e-4061-9ad7-fb5bbdaceb99 None admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:20.363963 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56204ef8-7e1e-4061-9ad7-fb5bbdaceb99 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:20.364235 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56204ef8-7e1e-4061-9ad7-fb5bbdaceb99 None admin] PATH_INFO: `/v3/projects/b4e2148743504b999b0e30a1fb3e97c7/users/9c2db95bdf92491b81a58119944240a5/roles/cdfc87279ea24b65bae3961035133725` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:20.366425 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56204ef8-7e1e-4061-9ad7-fb5bbdaceb99 None admin] RBAC: Authorizing `identity:create_grant(project_id=b4e2148743504b999b0e30a1fb3e97c7, user_id=9c2db95bdf92491b81a58119944240a5, role_id=cdfc87279ea24b65bae3961035133725)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:20.367852 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56204ef8-7e1e-4061-9ad7-fb5bbdaceb99 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:20.380365 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 559/1123] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 27 04:10:20 2022] PUT /identity/v3/projects/b4e2148743504b999b0e30a1fb3e97c7/users/9c2db95bdf92491b81a58119944240a5/roles/cdfc87279ea24b65bae3961035133725 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:10:22.359899 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee63b83-5d66-4886-a9f9-8a845c57d3e1 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:22.360457 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee63b83-5d66-4886-a9f9-8a845c57d3e1 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:22.360457 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee63b83-5d66-4886-a9f9-8a845c57d3e1 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:22.361787 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 565/1124] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:10:22.367025 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55577fc4-b006-49e2-a112-06dde054df86 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:22.367434 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55577fc4-b006-49e2-a112-06dde054df86 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:22.367586 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55577fc4-b006-49e2-a112-06dde054df86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:22.386031 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-55577fc4-b006-49e2-a112-06dde054df86 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:10:22.387234 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-55577fc4-b006-49e2-a112-06dde054df86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:10:22.453156 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 560/1125] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:10:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:10:22.481507 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee63b83-5d66-4886-a9f9-8a845c57d3e1 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:22.580282 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee63b83-5d66-4886-a9f9-8a845c57d3e1 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:22.612738 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69cfef-20a5-492b-8554-01c1858f1ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:22.613296 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69cfef-20a5-492b-8554-01c1858f1ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:22.613579 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69cfef-20a5-492b-8554-01c1858f1ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:22.613848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69cfef-20a5-492b-8554-01c1858f1ce9 None admin] PATH_INFO: `/v3/roles/cdfc87279ea24b65bae3961035133725` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:22.615324 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d69cfef-20a5-492b-8554-01c1858f1ce9 None admin] RBAC: Authorizing `identity:get_role(role_id=cdfc87279ea24b65bae3961035133725)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:22.616685 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d69cfef-20a5-492b-8554-01c1858f1ce9 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:22.618040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 566/1126] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:10:22 2022] GET /identity/v3/roles/cdfc87279ea24b65bae3961035133725 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:22.624036 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55577fc4-b006-49e2-a112-06dde054df86 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:22.649031 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:22.723408 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55577fc4-b006-49e2-a112-06dde054df86 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:22.756376 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca8133c-f6e2-4f7b-945b-37f4e0bbcbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:22.757007 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8133c-f6e2-4f7b-945b-37f4e0bbcbc6 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:22.757284 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8133c-f6e2-4f7b-945b-37f4e0bbcbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:22.757556 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8133c-f6e2-4f7b-945b-37f4e0bbcbc6 None admin] PATH_INFO: `/v3/users/9c2db95bdf92491b81a58119944240a5` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:22.758722 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca8133c-f6e2-4f7b-945b-37f4e0bbcbc6 None admin] RBAC: Authorizing `identity:get_user(user_id=9c2db95bdf92491b81a58119944240a5)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:22.760145 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca8133c-f6e2-4f7b-945b-37f4e0bbcbc6 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:22.761722 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 561/1127] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:10:22 2022] GET /identity/v3/users/9c2db95bdf92491b81a58119944240a5 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:22.767601 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69cfef-20a5-492b-8554-01c1858f1ce9 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:22.788447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:22.864766 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69cfef-20a5-492b-8554-01c1858f1ce9 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:22.897503 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24581ee6-c54a-4eb8-85fd-11c1721486dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:22.898166 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24581ee6-c54a-4eb8-85fd-11c1721486dc None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:22.898450 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24581ee6-c54a-4eb8-85fd-11c1721486dc None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:22.898724 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24581ee6-c54a-4eb8-85fd-11c1721486dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:22.962886 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24581ee6-c54a-4eb8-85fd-11c1721486dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:22.964328 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24581ee6-c54a-4eb8-85fd-11c1721486dc None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:22.984060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:23.058693 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 567/1128] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:10:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 04:10:23.065434 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca8133c-f6e2-4f7b-945b-37f4e0bbcbc6 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:23.154162 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:23.163838 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca8133c-f6e2-4f7b-945b-37f4e0bbcbc6 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:23.197127 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a3b2da-13b0-4022-8c44-7f7fadbfa0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:23.197768 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3b2da-13b0-4022-8c44-7f7fadbfa0e2 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:23.198088 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3b2da-13b0-4022-8c44-7f7fadbfa0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:23.198368 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3b2da-13b0-4022-8c44-7f7fadbfa0e2 None admin] PATH_INFO: `/v3/projects/b4e2148743504b999b0e30a1fb3e97c7` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:23.199523 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a3b2da-13b0-4022-8c44-7f7fadbfa0e2 None admin] RBAC: Authorizing `identity:get_project(project_id=b4e2148743504b999b0e30a1fb3e97c7)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:23.200950 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a3b2da-13b0-4022-8c44-7f7fadbfa0e2 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:23.202659 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 562/1129] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:10:23 2022] GET /identity/v3/projects/b4e2148743504b999b0e30a1fb3e97c7 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:23.208755 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24581ee6-c54a-4eb8-85fd-11c1721486dc None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:23.303914 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24581ee6-c54a-4eb8-85fd-11c1721486dc None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:23.322937 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:23.336717 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb1af1-8b15-4215-9eaf-7eb671780ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:23.337320 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb1af1-8b15-4215-9eaf-7eb671780ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:23.337597 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb1af1-8b15-4215-9eaf-7eb671780ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:23.337867 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb1af1-8b15-4215-9eaf-7eb671780ca3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:23.338686 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bb1af1-8b15-4215-9eaf-7eb671780ca3 None admin] RBAC: Adding query filter params (role.id=cdfc87279ea24b65bae3961035133725, scope.project.id=b4e2148743504b999b0e30a1fb3e97c7, user.id=9c2db95bdf92491b81a58119944240a5) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:23.339017 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bb1af1-8b15-4215-9eaf-7eb671780ca3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:23.340426 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bb1af1-8b15-4215-9eaf-7eb671780ca3 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:23.346629 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 568/1130] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:10:23 2022] GET /identity/v3/role_assignments?user.id=9c2db95bdf92491b81a58119944240a5&scope.project.id=b4e2148743504b999b0e30a1fb3e97c7&role.id=cdfc87279ea24b65bae3961035133725 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:25.321965 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d66d5f-5435-4ca9-9013-ac528a415c7c None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:25.322632 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d66d5f-5435-4ca9-9013-ac528a415c7c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:25.322632 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d66d5f-5435-4ca9-9013-ac528a415c7c None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:25.323893 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 563/1131] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:10:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:10:25.329732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc3735-60d1-4e6f-84c6-c99525662b00 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:25.330156 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc3735-60d1-4e6f-84c6-c99525662b00 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:25.330316 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc3735-60d1-4e6f-84c6-c99525662b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:25.348365 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-0ddc3735-60d1-4e6f-84c6-c99525662b00 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:10:25.349537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-0ddc3735-60d1-4e6f-84c6-c99525662b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:10:25.412634 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 569/1132] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:10:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 04:10:25.434820 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d66d5f-5435-4ca9-9013-ac528a415c7c None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:25.535281 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d66d5f-5435-4ca9-9013-ac528a415c7c None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:25.568620 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b08b1-a6b6-4571-9218-5b859783507a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:25.569257 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b08b1-a6b6-4571-9218-5b859783507a None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:25.569543 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b08b1-a6b6-4571-9218-5b859783507a None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:25.569819 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b08b1-a6b6-4571-9218-5b859783507a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:25.571001 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b08b1-a6b6-4571-9218-5b859783507a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:25.572480 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b08b1-a6b6-4571-9218-5b859783507a None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:25.574030 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 564/1133] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:10:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:25.580659 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc3735-60d1-4e6f-84c6-c99525662b00 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:25.673835 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc3735-60d1-4e6f-84c6-c99525662b00 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:25.688226 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:25.708668 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91504b6-e72f-4937-9ea6-de6cd75a8bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:25.709275 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91504b6-e72f-4937-9ea6-de6cd75a8bdb None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:25.709553 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91504b6-e72f-4937-9ea6-de6cd75a8bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:25.709825 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91504b6-e72f-4937-9ea6-de6cd75a8bdb None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:25.710535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91504b6-e72f-4937-9ea6-de6cd75a8bdb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:25.711949 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91504b6-e72f-4937-9ea6-de6cd75a8bdb None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:25.736828 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 570/1134] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Jul 27 04:10:25 2022] POST /identity/v3/projects => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:10:27.724178 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eec68c-01f7-457f-8ee0-50cafa21b371 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:27.724681 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eec68c-01f7-457f-8ee0-50cafa21b371 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:27.724681 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eec68c-01f7-457f-8ee0-50cafa21b371 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:27.726020 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 565/1135] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:10:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:10:27.731146 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083d5c3-1d1a-468e-a7c7-2ff8f62f462b None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:27.731554 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083d5c3-1d1a-468e-a7c7-2ff8f62f462b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:27.731698 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083d5c3-1d1a-468e-a7c7-2ff8f62f462b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:27.750917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-f083d5c3-1d1a-468e-a7c7-2ff8f62f462b None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:10:27.752098 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f083d5c3-1d1a-468e-a7c7-2ff8f62f462b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:10:27.817625 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 571/1136] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:10:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:10:27.841359 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eec68c-01f7-457f-8ee0-50cafa21b371 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:27.871812 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:27.945524 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eec68c-01f7-457f-8ee0-50cafa21b371 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:27.979699 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c7cc95-fa42-4828-8757-e5a227185600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:27.980265 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7cc95-fa42-4828-8757-e5a227185600 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:27.980543 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7cc95-fa42-4828-8757-e5a227185600 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:27.980816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7cc95-fa42-4828-8757-e5a227185600 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:27.981955 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c7cc95-fa42-4828-8757-e5a227185600 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:27.984490 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c7cc95-fa42-4828-8757-e5a227185600 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:27.986083 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 566/1137] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:10:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:27.991737 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083d5c3-1d1a-468e-a7c7-2ff8f62f462b None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:28.080698 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:28.086855 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083d5c3-1d1a-468e-a7c7-2ff8f62f462b None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:28.120691 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7125a2ee-8556-4d25-92d7-bc53d0e2cc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:28.121311 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125a2ee-8556-4d25-92d7-bc53d0e2cc42 None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:28.121593 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125a2ee-8556-4d25-92d7-bc53d0e2cc42 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:28.121869 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125a2ee-8556-4d25-92d7-bc53d0e2cc42 None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:28.122602 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7125a2ee-8556-4d25-92d7-bc53d0e2cc42 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:28.124040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7125a2ee-8556-4d25-92d7-bc53d0e2cc42 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:28.144570 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 572/1138] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 27 04:10:27 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:10:30.182088 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af163144-dbe2-4d1b-848b-21b040a3c690 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:30.182651 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af163144-dbe2-4d1b-848b-21b040a3c690 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:30.182651 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af163144-dbe2-4d1b-848b-21b040a3c690 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:30.184172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 567/1139] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:10:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:10:30.190650 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db203a-a085-46c7-b2a7-f154d4df9936 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:30.191093 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db203a-a085-46c7-b2a7-f154d4df9936 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:30.191204 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db203a-a085-46c7-b2a7-f154d4df9936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:30.209348 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-48db203a-a085-46c7-b2a7-f154d4df9936 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:10:30.210555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-48db203a-a085-46c7-b2a7-f154d4df9936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:10:30.274361 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 573/1140] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:10:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:10:30.297878 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af163144-dbe2-4d1b-848b-21b040a3c690 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:30.391269 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:30.398011 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af163144-dbe2-4d1b-848b-21b040a3c690 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:30.431311 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8990f62-1fea-4731-af24-e7fa9888c7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:30.431938 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8990f62-1fea-4731-af24-e7fa9888c7ad None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:30.432268 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8990f62-1fea-4731-af24-e7fa9888c7ad None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:30.432491 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8990f62-1fea-4731-af24-e7fa9888c7ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:30.443292 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8990f62-1fea-4731-af24-e7fa9888c7ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:10:30.448125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-d8990f62-1fea-4731-af24-e7fa9888c7ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:10:30.448125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:10:30.448125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:10:30.448125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:10:30.448125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:10:30.448125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:10:30.448125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:10:30.448125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:10:30.448125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:10:30.448125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:10:30.448125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:10:30.448125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:10:30.448125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:10:30.448125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:10:30.448125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:10:30.448125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:10:30.449401 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:10:30.449401 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:10:30.449401 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:10:30.449401 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:10:30.449401 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:10:30.449401 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:10:30.449401 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:10:30.449401 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:10:30.449401 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:10:30.449401 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:10:30.449401 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:10:30.449401 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:10:30.449401 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:10:30.449401 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:10:30.449401 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:10:30.449401 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:10:30.450477 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:10:30.450477 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:10:30.450477 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:10:30.450477 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:10:30.450477 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:10:30.450477 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:10:30.450477 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:10:30.450477 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:10:30.450477 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:10:30.450477 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:10:30.450477 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:10:30.450477 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:10:30.450477 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 568/1141] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:10:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:10:30.455548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db203a-a085-46c7-b2a7-f154d4df9936 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:30.532473 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:30.551428 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db203a-a085-46c7-b2a7-f154d4df9936 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:30.584822 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ce3e2-ec17-4ab5-82c7-c78b628c77b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:30.585447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ce3e2-ec17-4ab5-82c7-c78b628c77b4 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:30.585733 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ce3e2-ec17-4ab5-82c7-c78b628c77b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:30.586033 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ce3e2-ec17-4ab5-82c7-c78b628c77b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:30.586761 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918ce3e2-ec17-4ab5-82c7-c78b628c77b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:30.587068 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918ce3e2-ec17-4ab5-82c7-c78b628c77b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:30.588431 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918ce3e2-ec17-4ab5-82c7-c78b628c77b4 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:30.596314 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 574/1142] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:10:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:30.602043 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8990f62-1fea-4731-af24-e7fa9888c7ad None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:30.698985 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8990f62-1fea-4731-af24-e7fa9888c7ad None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:30.731938 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834d761-0fc9-4066-a217-60026444a496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:30.732565 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834d761-0fc9-4066-a217-60026444a496 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:30.732846 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834d761-0fc9-4066-a217-60026444a496 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:30.733120 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834d761-0fc9-4066-a217-60026444a496 None admin] PATH_INFO: `/v3/users/943bb1c43b2b47a988653ae99577fa17` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:30.746406 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5834d761-0fc9-4066-a217-60026444a496 None admin] RBAC: Authorizing `identity:get_user(user_id=943bb1c43b2b47a988653ae99577fa17)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:30.747825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5834d761-0fc9-4066-a217-60026444a496 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:30.749443 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 569/1143] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:10:30 2022] GET /identity/v3/users/943bb1c43b2b47a988653ae99577fa17 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:30.755358 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ce3e2-ec17-4ab5-82c7-c78b628c77b4 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:30.852638 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ce3e2-ec17-4ab5-82c7-c78b628c77b4 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:30.884919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0dfad-ccae-48be-b644-c547643c3431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:30.885528 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0dfad-ccae-48be-b644-c547643c3431 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:30.885807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0dfad-ccae-48be-b644-c547643c3431 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:30.886111 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0dfad-ccae-48be-b644-c547643c3431 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:30.899658 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:30.951858 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e0dfad-ccae-48be-b644-c547643c3431 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:30.953381 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e0dfad-ccae-48be-b644-c547643c3431 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:31.047206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 575/1144] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:10:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 04:10:31.053019 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834d761-0fc9-4066-a217-60026444a496 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:31.068334 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:31.151631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834d761-0fc9-4066-a217-60026444a496 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:31.184786 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19fa03-54e1-4d35-9603-7d53f44acc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:31.185418 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19fa03-54e1-4d35-9603-7d53f44acc1f None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:31.185694 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19fa03-54e1-4d35-9603-7d53f44acc1f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:31.185964 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19fa03-54e1-4d35-9603-7d53f44acc1f None admin] PATH_INFO: `/v3/projects/a2e6376d60f94b7f85ecb9b17d5be7fc` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:31.187184 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19fa03-54e1-4d35-9603-7d53f44acc1f None admin] RBAC: Authorizing `identity:get_project(project_id=a2e6376d60f94b7f85ecb9b17d5be7fc)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:31.188856 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19fa03-54e1-4d35-9603-7d53f44acc1f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:31.190522 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 570/1145] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:10:31 2022] GET /identity/v3/projects/a2e6376d60f94b7f85ecb9b17d5be7fc => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:31.196690 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0dfad-ccae-48be-b644-c547643c3431 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:31.241502 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:31.293731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0dfad-ccae-48be-b644-c547643c3431 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:31.326235 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e03c2-b770-42c0-b461-1449cc532cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:31.326849 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e03c2-b770-42c0-b461-1449cc532cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:31.327695 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e03c2-b770-42c0-b461-1449cc532cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:31.328178 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e03c2-b770-42c0-b461-1449cc532cc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:31.328974 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e03c2-b770-42c0-b461-1449cc532cc4 None admin] RBAC: Adding query filter params (role.id=cf771a8e94ee46149eb081347fea6d67, scope.project.id=a2e6376d60f94b7f85ecb9b17d5be7fc, user.id=943bb1c43b2b47a988653ae99577fa17) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:31.329288 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e03c2-b770-42c0-b461-1449cc532cc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:31.330731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e03c2-b770-42c0-b461-1449cc532cc4 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:31.335974 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 576/1146] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:10:31 2022] GET /identity/v3/role_assignments?user.id=943bb1c43b2b47a988653ae99577fa17&scope.project.id=a2e6376d60f94b7f85ecb9b17d5be7fc&role.id=cf771a8e94ee46149eb081347fea6d67 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:33.271577 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40c326-6043-4d96-9703-212800fc15a2 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:33.272122 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40c326-6043-4d96-9703-212800fc15a2 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:33.272122 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40c326-6043-4d96-9703-212800fc15a2 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:33.273456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 571/1147] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:10:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:10:33.278659 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00bb82-2b13-49a7-95b9-e5deccb0115c None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:33.279036 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00bb82-2b13-49a7-95b9-e5deccb0115c None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:33.279215 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00bb82-2b13-49a7-95b9-e5deccb0115c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:33.297461 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-3d00bb82-2b13-49a7-95b9-e5deccb0115c None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:10:33.298672 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-3d00bb82-2b13-49a7-95b9-e5deccb0115c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:10:33.363043 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 577/1148] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:10:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:10:33.385423 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40c326-6043-4d96-9703-212800fc15a2 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:33.468405 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:33.487717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40c326-6043-4d96-9703-212800fc15a2 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:33.520545 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523aa593-8d89-4f45-a64b-e024ae397e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:33.521170 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523aa593-8d89-4f45-a64b-e024ae397e48 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:33.521446 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523aa593-8d89-4f45-a64b-e024ae397e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:33.521762 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523aa593-8d89-4f45-a64b-e024ae397e48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:33.531586 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523aa593-8d89-4f45-a64b-e024ae397e48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:10:33.536388 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-523aa593-8d89-4f45-a64b-e024ae397e48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:10:33.536388 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:10:33.536388 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:10:33.536388 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:10:33.536388 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:10:33.536388 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:10:33.536388 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:10:33.536388 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:10:33.536388 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:10:33.536388 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:10:33.536388 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:10:33.536388 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:10:33.536388 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:10:33.536388 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:10:33.536388 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:10:33.536388 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:10:33.537497 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:10:33.537497 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:10:33.537497 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:10:33.537497 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:10:33.537497 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:10:33.537497 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:10:33.537497 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:10:33.537497 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:10:33.537497 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:10:33.537497 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:10:33.537497 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:10:33.537497 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:10:33.537497 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:10:33.537497 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:10:33.537497 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:10:33.537497 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:10:33.538459 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:10:33.538459 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:10:33.538459 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:10:33.538459 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:10:33.538459 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:10:33.538459 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:10:33.538459 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:10:33.538459 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:10:33.538459 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:10:33.538459 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:10:33.538459 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:10:33.538459 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:10:33.538459 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 572/1149] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:10:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:10:33.543543 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d00bb82-2b13-49a7-95b9-e5deccb0115c None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:33.582260 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:33.640408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d00bb82-2b13-49a7-95b9-e5deccb0115c None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:33.674137 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42cd84f-4dbf-44ac-b581-6c6d599c2248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:33.674747 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42cd84f-4dbf-44ac-b581-6c6d599c2248 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:33.675029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42cd84f-4dbf-44ac-b581-6c6d599c2248 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:33.675303 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42cd84f-4dbf-44ac-b581-6c6d599c2248 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:33.676016 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42cd84f-4dbf-44ac-b581-6c6d599c2248 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:33.676320 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42cd84f-4dbf-44ac-b581-6c6d599c2248 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:33.677679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42cd84f-4dbf-44ac-b581-6c6d599c2248 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:33.685671 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 578/1150] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:10:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:33.692548 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523aa593-8d89-4f45-a64b-e024ae397e48 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:33.789665 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523aa593-8d89-4f45-a64b-e024ae397e48 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:33.823042 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9b22c-5696-43a5-8f70-383b55b45765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:33.823663 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9b22c-5696-43a5-8f70-383b55b45765 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:33.823940 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9b22c-5696-43a5-8f70-383b55b45765 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:33.824210 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9b22c-5696-43a5-8f70-383b55b45765 None admin] PATH_INFO: `/v3/users/943bb1c43b2b47a988653ae99577fa17` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:33.825342 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c9b22c-5696-43a5-8f70-383b55b45765 None admin] RBAC: Authorizing `identity:get_user(user_id=943bb1c43b2b47a988653ae99577fa17)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:33.826785 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c9b22c-5696-43a5-8f70-383b55b45765 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:33.828672 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 573/1151] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:10:33 2022] GET /identity/v3/users/943bb1c43b2b47a988653ae99577fa17 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:10:33.834409 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42cd84f-4dbf-44ac-b581-6c6d599c2248 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:33.936451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:33.939758 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42cd84f-4dbf-44ac-b581-6c6d599c2248 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:33.977618 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d81756-e790-427f-99e2-edf48748d194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:33.978295 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d81756-e790-427f-99e2-edf48748d194 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:33.978579 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d81756-e790-427f-99e2-edf48748d194 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:33.978857 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d81756-e790-427f-99e2-edf48748d194 None admin] PATH_INFO: `/v3/projects/a2e6376d60f94b7f85ecb9b17d5be7fc` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:33.980034 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d81756-e790-427f-99e2-edf48748d194 None admin] RBAC: Authorizing `identity:get_project(project_id=a2e6376d60f94b7f85ecb9b17d5be7fc)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:33.981473 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d81756-e790-427f-99e2-edf48748d194 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:33.983217 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 579/1152] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:10:33 2022] GET /identity/v3/projects/a2e6376d60f94b7f85ecb9b17d5be7fc => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:33.989795 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9b22c-5696-43a5-8f70-383b55b45765 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:34.002700 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:34.087324 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9b22c-5696-43a5-8f70-383b55b45765 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:34.121552 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0fb6ba-5f79-432d-8376-dd1a9d984701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:34.122250 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0fb6ba-5f79-432d-8376-dd1a9d984701 None admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:34.122526 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0fb6ba-5f79-432d-8376-dd1a9d984701 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:34.122794 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0fb6ba-5f79-432d-8376-dd1a9d984701 None admin] PATH_INFO: `/v3/projects/a2e6376d60f94b7f85ecb9b17d5be7fc/users/943bb1c43b2b47a988653ae99577fa17/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:34.124826 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0fb6ba-5f79-432d-8376-dd1a9d984701 None admin] RBAC: Authorizing `identity:create_grant(project_id=a2e6376d60f94b7f85ecb9b17d5be7fc, user_id=943bb1c43b2b47a988653ae99577fa17, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:34.126271 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0fb6ba-5f79-432d-8376-dd1a9d984701 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:34.140210 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 574/1153] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 27 04:10:33 2022] PUT /identity/v3/projects/a2e6376d60f94b7f85ecb9b17d5be7fc/users/943bb1c43b2b47a988653ae99577fa17/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:10:36.140103 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd704e4-1d30-4edd-84aa-5ef08798d6db None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:36.140629 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd704e4-1d30-4edd-84aa-5ef08798d6db None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:36.140629 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd704e4-1d30-4edd-84aa-5ef08798d6db None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:36.142007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 580/1154] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:10:36.147162 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b3b62-5a2e-422e-b40c-11efc9de289c None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:36.147580 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b3b62-5a2e-422e-b40c-11efc9de289c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:36.147718 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b3b62-5a2e-422e-b40c-11efc9de289c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:36.165768 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-ed2b3b62-5a2e-422e-b40c-11efc9de289c None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:10:36.166987 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ed2b3b62-5a2e-422e-b40c-11efc9de289c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:10:36.227358 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:36.232818 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 575/1155] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:10:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:10:36.255992 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd704e4-1d30-4edd-84aa-5ef08798d6db None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:36.355549 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd704e4-1d30-4edd-84aa-5ef08798d6db None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:36.387082 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adeba60-4e17-4472-a0cd-ee7e5b5672af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:36.387708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adeba60-4e17-4472-a0cd-ee7e5b5672af None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:36.387989 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adeba60-4e17-4472-a0cd-ee7e5b5672af None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:36.388258 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adeba60-4e17-4472-a0cd-ee7e5b5672af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:36.399719 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adeba60-4e17-4472-a0cd-ee7e5b5672af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:10:36.404471 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-2adeba60-4e17-4472-a0cd-ee7e5b5672af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:10:36.404471 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:10:36.404471 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:10:36.404471 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:10:36.404471 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:10:36.404471 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:10:36.404471 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:10:36.404471 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:10:36.404471 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:10:36.404471 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:10:36.404471 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:10:36.404471 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:10:36.404471 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:10:36.404471 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:10:36.404471 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:10:36.404471 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:10:36.405563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:10:36.405563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:10:36.405563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:10:36.405563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:10:36.405563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:10:36.405563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:10:36.405563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:10:36.405563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:10:36.405563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:10:36.405563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:10:36.405563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:10:36.405563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:10:36.405563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:10:36.405563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:10:36.405563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:10:36.405563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:10:36.406616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:10:36.406616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:10:36.406616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:10:36.406616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:10:36.406616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:10:36.406616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:10:36.406616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:10:36.406616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:10:36.406616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:10:36.406616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:10:36.406616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:10:36.406616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:10:36.406616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 581/1156] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:10:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:10:36.411882 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b3b62-5a2e-422e-b40c-11efc9de289c None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:36.509425 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b3b62-5a2e-422e-b40c-11efc9de289c None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:36.542227 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b6b00-a982-4fcb-b582-bc0562eda541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:36.542853 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b6b00-a982-4fcb-b582-bc0562eda541 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:36.543135 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b6b00-a982-4fcb-b582-bc0562eda541 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:36.543409 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b6b00-a982-4fcb-b582-bc0562eda541 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:36.544130 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b6b00-a982-4fcb-b582-bc0562eda541 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:36.544435 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b6b00-a982-4fcb-b582-bc0562eda541 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:36.546342 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b6b00-a982-4fcb-b582-bc0562eda541 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:36.554142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 576/1157] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:10:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:36.559937 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adeba60-4e17-4472-a0cd-ee7e5b5672af None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:36.587809 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:36.657501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adeba60-4e17-4472-a0cd-ee7e5b5672af None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:36.689995 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f91ec2-f408-4b6a-9f84-5202936f5766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:36.690919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f91ec2-f408-4b6a-9f84-5202936f5766 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:36.691200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f91ec2-f408-4b6a-9f84-5202936f5766 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:36.691472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f91ec2-f408-4b6a-9f84-5202936f5766 None admin] PATH_INFO: `/v3/users/943bb1c43b2b47a988653ae99577fa17` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:36.692619 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f91ec2-f408-4b6a-9f84-5202936f5766 None admin] RBAC: Authorizing `identity:get_user(user_id=943bb1c43b2b47a988653ae99577fa17)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:36.694054 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f91ec2-f408-4b6a-9f84-5202936f5766 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:36.695644 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 582/1158] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:10:36 2022] GET /identity/v3/users/943bb1c43b2b47a988653ae99577fa17 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:36.701455 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b6b00-a982-4fcb-b582-bc0562eda541 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:36.761513 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:36.799899 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b6b00-a982-4fcb-b582-bc0562eda541 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:36.832263 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d04e46-1435-48b6-8abb-93aab4905a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:36.832879 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d04e46-1435-48b6-8abb-93aab4905a8a None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:36.833159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d04e46-1435-48b6-8abb-93aab4905a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:36.833433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d04e46-1435-48b6-8abb-93aab4905a8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:36.897711 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d04e46-1435-48b6-8abb-93aab4905a8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:36.899196 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d04e46-1435-48b6-8abb-93aab4905a8a None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:36.961934 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:37.001610 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 577/1159] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:10:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 04:10:37.007609 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f91ec2-f408-4b6a-9f84-5202936f5766 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:37.096366 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:37.102795 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f91ec2-f408-4b6a-9f84-5202936f5766 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:37.135947 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dcfaf4-3295-4e3e-af08-61c9fbcea4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:37.136584 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dcfaf4-3295-4e3e-af08-61c9fbcea4dc None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:37.136870 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dcfaf4-3295-4e3e-af08-61c9fbcea4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:37.137148 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dcfaf4-3295-4e3e-af08-61c9fbcea4dc None admin] PATH_INFO: `/v3/projects/a2e6376d60f94b7f85ecb9b17d5be7fc` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:37.138335 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dcfaf4-3295-4e3e-af08-61c9fbcea4dc None admin] RBAC: Authorizing `identity:get_project(project_id=a2e6376d60f94b7f85ecb9b17d5be7fc)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:37.139813 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dcfaf4-3295-4e3e-af08-61c9fbcea4dc None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:37.141552 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 583/1160] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:10:37 2022] GET /identity/v3/projects/a2e6376d60f94b7f85ecb9b17d5be7fc => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:37.147568 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d04e46-1435-48b6-8abb-93aab4905a8a None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:37.243344 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d04e46-1435-48b6-8abb-93aab4905a8a None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:37.276002 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1655a5b-93d1-49ea-8fef-2baecc20872b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:37.276608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1655a5b-93d1-49ea-8fef-2baecc20872b None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:37.276885 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1655a5b-93d1-49ea-8fef-2baecc20872b None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:37.277156 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1655a5b-93d1-49ea-8fef-2baecc20872b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:37.277928 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1655a5b-93d1-49ea-8fef-2baecc20872b None admin] RBAC: Adding query filter params (role.id=cf771a8e94ee46149eb081347fea6d67, scope.project.id=a2e6376d60f94b7f85ecb9b17d5be7fc, user.id=943bb1c43b2b47a988653ae99577fa17) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:37.278263 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1655a5b-93d1-49ea-8fef-2baecc20872b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:37.279690 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1655a5b-93d1-49ea-8fef-2baecc20872b None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:37.285310 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 578/1161] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:10:37 2022] GET /identity/v3/role_assignments?user.id=943bb1c43b2b47a988653ae99577fa17&scope.project.id=a2e6376d60f94b7f85ecb9b17d5be7fc&role.id=cf771a8e94ee46149eb081347fea6d67 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:39.267773 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8abd3-b344-4c9b-b0c1-17f3ed9f952f None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:39.268340 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8abd3-b344-4c9b-b0c1-17f3ed9f952f None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:39.268420 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8abd3-b344-4c9b-b0c1-17f3ed9f952f None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:39.269730 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 584/1162] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:10:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 04:10:39.274998 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5149460-fc06-4d1b-82da-ee048f1ab52d None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:39.275386 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5149460-fc06-4d1b-82da-ee048f1ab52d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:39.275552 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5149460-fc06-4d1b-82da-ee048f1ab52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:39.293810 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-c5149460-fc06-4d1b-82da-ee048f1ab52d None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:10:39.295025 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c5149460-fc06-4d1b-82da-ee048f1ab52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:10:39.314705 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:39.360845 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 579/1163] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:10:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:10:39.382595 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8abd3-b344-4c9b-b0c1-17f3ed9f952f None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:39.485841 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8abd3-b344-4c9b-b0c1-17f3ed9f952f None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:39.519455 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a2f7f-eef1-41dc-8b0d-5c09e379ca05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:39.520018 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a2f7f-eef1-41dc-8b0d-5c09e379ca05 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:39.520304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a2f7f-eef1-41dc-8b0d-5c09e379ca05 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:39.520581 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a2f7f-eef1-41dc-8b0d-5c09e379ca05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:39.550814 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:39.587433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a2f7f-eef1-41dc-8b0d-5c09e379ca05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:39.588871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a2f7f-eef1-41dc-8b0d-5c09e379ca05 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:39.684344 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 585/1164] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:10:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 04:10:39.690688 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5149460-fc06-4d1b-82da-ee048f1ab52d None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:39.785219 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5149460-fc06-4d1b-82da-ee048f1ab52d None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:39.817820 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917af4f-5ac9-4b01-81e2-1cf68b58b5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:39.818484 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917af4f-5ac9-4b01-81e2-1cf68b58b5b9 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:39.818762 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917af4f-5ac9-4b01-81e2-1cf68b58b5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:39.819033 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917af4f-5ac9-4b01-81e2-1cf68b58b5b9 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:39.825440 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917af4f-5ac9-4b01-81e2-1cf68b58b5b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:39.827025 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917af4f-5ac9-4b01-81e2-1cf68b58b5b9 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:39.838070 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-7917af4f-5ac9-4b01-81e2-1cf68b58b5b9 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 27 04:10:39.838070 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:10:39.838070 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 27 04:10:39.838070 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 27 04:10:39.838070 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:10:39.838070 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:10:39.838070 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:10:39.838070 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:10:39.838070 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 27 04:10:39.838070 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:10:39.838070 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 04:10:39.838070 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:10:39.838070 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:10:39.838070 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:10:39.838070 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:10:39.838070 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:10:39.838070 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:10:39.839291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:10:39.839291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:10:39.839291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:10:39.839291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:10:39.839291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:10:39.839291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:10:39.839291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:10:39.839291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:10:39.839291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:10:39.839291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:10:39.839291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:10:39.839291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:10:39.839291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:10:39.839291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:10:39.839291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:10:39.839291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:10:39.840165 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:10:39.840165 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:10:39.840165 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:10:39.840165 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:10:39.840165 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:10:39.840165 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:10:39.840165 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:10:39.840165 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:10:39.840165 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:10:39.840165 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:10:39.840165 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 27 04:10:39.840165 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:10:39.840165 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 27 04:10:39.840165 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:10:39.840165 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 580/1165] 10.222.0.46 () {56 vars in 1117 bytes} [Wed Jul 27 04:10:39 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:10:39.846248 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a2f7f-eef1-41dc-8b0d-5c09e379ca05 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:39.906102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:39.942991 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a2f7f-eef1-41dc-8b0d-5c09e379ca05 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:39.975077 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0250890-043e-4b5d-8211-b1b88b48c788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:39.975684 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0250890-043e-4b5d-8211-b1b88b48c788 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:39.975964 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0250890-043e-4b5d-8211-b1b88b48c788 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:39.976238 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0250890-043e-4b5d-8211-b1b88b48c788 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:39.976922 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0250890-043e-4b5d-8211-b1b88b48c788 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:39.977224 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0250890-043e-4b5d-8211-b1b88b48c788 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:39.978602 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0250890-043e-4b5d-8211-b1b88b48c788 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:39.985989 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 586/1166] 10.222.0.46 () {56 vars in 1115 bytes} [Wed Jul 27 04:10:39 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:41.950086 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c2998-f3a8-46ee-9483-8870074fa3c7 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:41.950609 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c2998-f3a8-46ee-9483-8870074fa3c7 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:41.950609 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c2998-f3a8-46ee-9483-8870074fa3c7 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:41.952174 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 581/1167] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:10:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:10:41.957375 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccda9b5-918b-44b5-8726-7feace3888c7 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:41.957802 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccda9b5-918b-44b5-8726-7feace3888c7 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:41.957928 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccda9b5-918b-44b5-8726-7feace3888c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:41.976663 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-6ccda9b5-918b-44b5-8726-7feace3888c7 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:10:41.977839 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-6ccda9b5-918b-44b5-8726-7feace3888c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:10:42.041067 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 587/1168] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:10:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:10:42.064047 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c2998-f3a8-46ee-9483-8870074fa3c7 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:42.082264 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:42.163920 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c2998-f3a8-46ee-9483-8870074fa3c7 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:42.196921 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54ea317-5e4b-49a5-8902-877e708b4927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:42.197529 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ea317-5e4b-49a5-8902-877e708b4927 None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:42.197811 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ea317-5e4b-49a5-8902-877e708b4927 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:42.198109 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ea317-5e4b-49a5-8902-877e708b4927 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:42.198833 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54ea317-5e4b-49a5-8902-877e708b4927 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:42.200248 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54ea317-5e4b-49a5-8902-877e708b4927 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:42.215309 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 582/1169] 10.222.0.46 () {60 vars in 1138 bytes} [Wed Jul 27 04:10:42 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:10:44.230519 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24dc69-7ccb-4b08-b378-b52372004864 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:44.231084 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24dc69-7ccb-4b08-b378-b52372004864 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:44.231084 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24dc69-7ccb-4b08-b378-b52372004864 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:44.232433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 588/1170] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:10:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:10:44.237599 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926926cb-c5af-4467-b546-2b26adaab603 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:44.238022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926926cb-c5af-4467-b546-2b26adaab603 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:44.238208 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926926cb-c5af-4467-b546-2b26adaab603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:44.256415 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-926926cb-c5af-4467-b546-2b26adaab603 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:10:44.257594 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-926926cb-c5af-4467-b546-2b26adaab603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:10:44.327790 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 583/1171] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:10:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:10:44.349950 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24dc69-7ccb-4b08-b378-b52372004864 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:44.406967 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:44.447851 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24dc69-7ccb-4b08-b378-b52372004864 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:44.480751 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30bfc59-15f8-44bf-8edf-a0afbb9dd422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:44.481327 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30bfc59-15f8-44bf-8edf-a0afbb9dd422 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:44.481605 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30bfc59-15f8-44bf-8edf-a0afbb9dd422 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:44.481880 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30bfc59-15f8-44bf-8edf-a0afbb9dd422 None admin] PATH_INFO: `/v3/domains/187c79b927e3413cbef741af3d851b74` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:44.492334 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30bfc59-15f8-44bf-8edf-a0afbb9dd422 None admin] RBAC: Authorizing `identity:get_domain(domain_id=187c79b927e3413cbef741af3d851b74)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:44.493759 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30bfc59-15f8-44bf-8edf-a0afbb9dd422 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:44.495389 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 589/1172] 10.222.0.46 () {56 vars in 1183 bytes} [Wed Jul 27 04:10:44 2022] GET /identity/v3/domains/187c79b927e3413cbef741af3d851b74 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:44.501956 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926926cb-c5af-4467-b546-2b26adaab603 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:44.509822 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:44.598511 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926926cb-c5af-4467-b546-2b26adaab603 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:44.631784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a26e39-feab-4bde-9246-d20ecdbc8f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:44.632376 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a26e39-feab-4bde-9246-d20ecdbc8f8f None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:44.632654 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a26e39-feab-4bde-9246-d20ecdbc8f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:44.632921 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a26e39-feab-4bde-9246-d20ecdbc8f8f None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:44.633605 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a26e39-feab-4bde-9246-d20ecdbc8f8f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:44.635060 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a26e39-feab-4bde-9246-d20ecdbc8f8f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:44.660459 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 584/1173] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Jul 27 04:10:44 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:10:46.679313 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa0b8d-3646-485a-b03d-24a7a502b8a6 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:46.679885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa0b8d-3646-485a-b03d-24a7a502b8a6 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:46.679885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa0b8d-3646-485a-b03d-24a7a502b8a6 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:46.681216 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 590/1174] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:10:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:10:46.687120 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cabb9b-f38d-4e97-84a8-002d91a5e1ae None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:46.687524 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cabb9b-f38d-4e97-84a8-002d91a5e1ae None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:46.687721 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cabb9b-f38d-4e97-84a8-002d91a5e1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:46.706555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-97cabb9b-f38d-4e97-84a8-002d91a5e1ae None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:10:46.707778 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-97cabb9b-f38d-4e97-84a8-002d91a5e1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:10:46.760049 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:46.774817 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 585/1175] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:10:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:10:46.800165 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa0b8d-3646-485a-b03d-24a7a502b8a6 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:46.897756 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa0b8d-3646-485a-b03d-24a7a502b8a6 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:46.913701 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:46.930883 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941d7e4-a8dc-404e-b897-fe33ee63c374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:46.931516 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941d7e4-a8dc-404e-b897-fe33ee63c374 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:46.931794 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941d7e4-a8dc-404e-b897-fe33ee63c374 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:46.932069 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941d7e4-a8dc-404e-b897-fe33ee63c374 None admin] PATH_INFO: `/v3/domains/187c79b927e3413cbef741af3d851b74` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:46.933201 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2941d7e4-a8dc-404e-b897-fe33ee63c374 None admin] RBAC: Authorizing `identity:get_domain(domain_id=187c79b927e3413cbef741af3d851b74)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:46.934654 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2941d7e4-a8dc-404e-b897-fe33ee63c374 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:46.936901 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 591/1176] 10.222.0.46 () {56 vars in 1183 bytes} [Wed Jul 27 04:10:46 2022] GET /identity/v3/domains/187c79b927e3413cbef741af3d851b74 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:46.943000 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cabb9b-f38d-4e97-84a8-002d91a5e1ae None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:47.036763 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cabb9b-f38d-4e97-84a8-002d91a5e1ae None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:47.071425 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d91055-cd24-4749-9c31-caf18759ef57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:47.072031 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d91055-cd24-4749-9c31-caf18759ef57 None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:47.072308 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d91055-cd24-4749-9c31-caf18759ef57 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:47.072586 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d91055-cd24-4749-9c31-caf18759ef57 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:47.073268 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d91055-cd24-4749-9c31-caf18759ef57 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:47.074704 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d91055-cd24-4749-9c31-caf18759ef57 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:47.095604 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 586/1177] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 27 04:10:46 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:10:49.106581 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37efec72-ee12-48c5-b7c1-629eee7e4e47 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:49.107132 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37efec72-ee12-48c5-b7c1-629eee7e4e47 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:49.107132 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37efec72-ee12-48c5-b7c1-629eee7e4e47 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:49.108983 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 592/1178] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:10:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:10:49.114248 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f09cd-3a6d-433a-855f-546d85f8849a None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:49.114646 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f09cd-3a6d-433a-855f-546d85f8849a None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:49.114805 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f09cd-3a6d-433a-855f-546d85f8849a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:49.133010 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-b52f09cd-3a6d-433a-855f-546d85f8849a None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:10:49.134222 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b52f09cd-3a6d-433a-855f-546d85f8849a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:10:49.173845 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:49.202469 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 587/1179] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:10:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:10:49.225506 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37efec72-ee12-48c5-b7c1-629eee7e4e47 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:49.325436 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37efec72-ee12-48c5-b7c1-629eee7e4e47 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:49.356765 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc7c26-6684-4038-af98-286303ad38aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:49.357358 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc7c26-6684-4038-af98-286303ad38aa None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:49.357639 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc7c26-6684-4038-af98-286303ad38aa None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:49.358425 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc7c26-6684-4038-af98-286303ad38aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:49.368428 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc7c26-6684-4038-af98-286303ad38aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:10:49.373244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-3bcc7c26-6684-4038-af98-286303ad38aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:10:49.373244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:10:49.373244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:10:49.373244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:10:49.373244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:10:49.373244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:10:49.373244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:10:49.373244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:10:49.373244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:10:49.373244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:10:49.373244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:10:49.373244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:10:49.373244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:10:49.373244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:10:49.373244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:10:49.373244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:10:49.375030 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:10:49.375030 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:10:49.375030 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:10:49.375030 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:10:49.375030 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:10:49.375030 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:10:49.375030 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:10:49.375030 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:10:49.375030 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:10:49.375030 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:10:49.375030 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:10:49.375030 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:10:49.375030 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:10:49.375030 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:10:49.375030 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:10:49.375030 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:10:49.375917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:10:49.375917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:10:49.375917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:10:49.375917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:10:49.375917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:10:49.375917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:10:49.375917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:10:49.375917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:10:49.375917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:10:49.375917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:10:49.375917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:10:49.375917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:10:49.375917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 593/1180] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:10:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:10:49.381000 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f09cd-3a6d-433a-855f-546d85f8849a None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:49.480546 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f09cd-3a6d-433a-855f-546d85f8849a None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:49.513891 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea433a09-6b9b-4840-88b3-08e108af70f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:49.515440 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea433a09-6b9b-4840-88b3-08e108af70f7 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:49.515725 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea433a09-6b9b-4840-88b3-08e108af70f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:49.516001 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea433a09-6b9b-4840-88b3-08e108af70f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:49.516720 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea433a09-6b9b-4840-88b3-08e108af70f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:49.517023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea433a09-6b9b-4840-88b3-08e108af70f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:49.518423 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea433a09-6b9b-4840-88b3-08e108af70f7 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:49.526229 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 588/1181] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:10:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:49.532200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc7c26-6684-4038-af98-286303ad38aa None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:49.579296 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:49.631911 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc7c26-6684-4038-af98-286303ad38aa None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:49.664516 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba10a9-23b4-4fa8-a244-8be5f35a87ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:49.665146 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba10a9-23b4-4fa8-a244-8be5f35a87ce None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:49.665431 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba10a9-23b4-4fa8-a244-8be5f35a87ce None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:49.665709 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba10a9-23b4-4fa8-a244-8be5f35a87ce None admin] PATH_INFO: `/v3/users/ff7198f9399a45d9bca9609255e07e6c` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:49.680224 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba10a9-23b4-4fa8-a244-8be5f35a87ce None admin] RBAC: Authorizing `identity:get_user(user_id=ff7198f9399a45d9bca9609255e07e6c)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:49.682877 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba10a9-23b4-4fa8-a244-8be5f35a87ce None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:49.684426 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 594/1182] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:10:49 2022] GET /identity/v3/users/ff7198f9399a45d9bca9609255e07e6c => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:49.691512 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea433a09-6b9b-4840-88b3-08e108af70f7 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:49.727939 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:49.791526 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea433a09-6b9b-4840-88b3-08e108af70f7 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:49.825727 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c061fb-70e3-4e31-abfc-2ce22030b2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:49.826542 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c061fb-70e3-4e31-abfc-2ce22030b2d2 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:49.826821 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c061fb-70e3-4e31-abfc-2ce22030b2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:49.827272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c061fb-70e3-4e31-abfc-2ce22030b2d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:49.893660 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c061fb-70e3-4e31-abfc-2ce22030b2d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:49.895090 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c061fb-70e3-4e31-abfc-2ce22030b2d2 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:49.927170 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:49.990871 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 589/1183] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:10:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 04:10:49.996885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba10a9-23b4-4fa8-a244-8be5f35a87ce None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:50.097967 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba10a9-23b4-4fa8-a244-8be5f35a87ce None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:50.100254 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:50.130881 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8f0e6-6bc0-46fe-a9fa-13b342f36d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:50.131516 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8f0e6-6bc0-46fe-a9fa-13b342f36d43 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:50.131794 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8f0e6-6bc0-46fe-a9fa-13b342f36d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:50.132094 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8f0e6-6bc0-46fe-a9fa-13b342f36d43 None admin] PATH_INFO: `/v3/projects/91ca556673604d438fcf0945ec430937` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:50.133233 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8f0e6-6bc0-46fe-a9fa-13b342f36d43 None admin] RBAC: Authorizing `identity:get_project(project_id=91ca556673604d438fcf0945ec430937)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:50.134690 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8f0e6-6bc0-46fe-a9fa-13b342f36d43 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:50.136336 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 595/1184] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:10:49 2022] GET /identity/v3/projects/91ca556673604d438fcf0945ec430937 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:50.143215 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c061fb-70e3-4e31-abfc-2ce22030b2d2 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:50.237786 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c061fb-70e3-4e31-abfc-2ce22030b2d2 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:50.270263 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3060496d-3c74-4e0a-878a-958abd7b9c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:50.270864 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3060496d-3c74-4e0a-878a-958abd7b9c2f None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:50.271142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3060496d-3c74-4e0a-878a-958abd7b9c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:50.271411 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3060496d-3c74-4e0a-878a-958abd7b9c2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:50.272189 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3060496d-3c74-4e0a-878a-958abd7b9c2f None admin] RBAC: Adding query filter params (role.id=cf771a8e94ee46149eb081347fea6d67, scope.project.id=91ca556673604d438fcf0945ec430937, user.id=ff7198f9399a45d9bca9609255e07e6c) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:50.272492 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3060496d-3c74-4e0a-878a-958abd7b9c2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:50.273923 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3060496d-3c74-4e0a-878a-958abd7b9c2f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:50.276723 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:50.282325 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 590/1185] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:10:50 2022] GET /identity/v3/role_assignments?user.id=ff7198f9399a45d9bca9609255e07e6c&scope.project.id=91ca556673604d438fcf0945ec430937&role.id=cf771a8e94ee46149eb081347fea6d67 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:52.225116 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880dcd24-6242-4441-b395-8a3ce35560d3 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:52.225670 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880dcd24-6242-4441-b395-8a3ce35560d3 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:52.225670 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880dcd24-6242-4441-b395-8a3ce35560d3 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:52.227034 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 596/1186] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:10:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:10:52.232090 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ab818-4e16-45c3-b31d-994c4c7d1e7a None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:52.232482 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ab818-4e16-45c3-b31d-994c4c7d1e7a None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:52.232634 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ab818-4e16-45c3-b31d-994c4c7d1e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:52.250272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-644ab818-4e16-45c3-b31d-994c4c7d1e7a None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:10:52.251439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-644ab818-4e16-45c3-b31d-994c4c7d1e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:10:52.314675 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 591/1187] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:10:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:10:52.336565 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880dcd24-6242-4441-b395-8a3ce35560d3 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:52.433859 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880dcd24-6242-4441-b395-8a3ce35560d3 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:52.466251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60c5bd-d40f-4496-b779-7b5b688f08cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:52.466885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60c5bd-d40f-4496-b779-7b5b688f08cd None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:52.467705 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60c5bd-d40f-4496-b779-7b5b688f08cd None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:52.467974 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60c5bd-d40f-4496-b779-7b5b688f08cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:52.477737 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e60c5bd-d40f-4496-b779-7b5b688f08cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:10:52.482538 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-3e60c5bd-d40f-4496-b779-7b5b688f08cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:10:52.482538 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:10:52.482538 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:10:52.482538 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:10:52.482538 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:10:52.482538 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:10:52.482538 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:10:52.482538 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:10:52.482538 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:10:52.482538 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:10:52.482538 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:10:52.482538 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:10:52.482538 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:10:52.482538 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:10:52.482538 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:10:52.482538 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:10:52.484406 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:10:52.484406 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:10:52.484406 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:10:52.484406 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:10:52.484406 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:10:52.484406 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:10:52.484406 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:10:52.484406 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:10:52.484406 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:10:52.484406 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:10:52.484406 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:10:52.484406 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:10:52.484406 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:10:52.484406 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:10:52.484406 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:10:52.484406 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:10:52.485686 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:10:52.485686 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:10:52.485686 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:10:52.485686 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:10:52.485686 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:10:52.485686 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:10:52.485686 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:10:52.485686 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:10:52.485686 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:10:52.485686 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:10:52.485686 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:10:52.485686 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:10:52.485686 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 597/1188] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:10:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:10:52.490161 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ab818-4e16-45c3-b31d-994c4c7d1e7a None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:52.585836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ab818-4e16-45c3-b31d-994c4c7d1e7a None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:52.616858 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:52.622765 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14dd572-8c9f-4442-945d-07bc5c1f0119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:52.623945 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14dd572-8c9f-4442-945d-07bc5c1f0119 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:52.624222 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14dd572-8c9f-4442-945d-07bc5c1f0119 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:52.624491 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14dd572-8c9f-4442-945d-07bc5c1f0119 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:52.625205 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14dd572-8c9f-4442-945d-07bc5c1f0119 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:52.625503 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14dd572-8c9f-4442-945d-07bc5c1f0119 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:52.626895 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14dd572-8c9f-4442-945d-07bc5c1f0119 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:52.634727 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 592/1189] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:10:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:52.641143 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60c5bd-d40f-4496-b779-7b5b688f08cd None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:52.671122 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:52.737864 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60c5bd-d40f-4496-b779-7b5b688f08cd None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:52.770076 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f44d71a-1348-4419-af5d-0156c65aef6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:52.770701 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44d71a-1348-4419-af5d-0156c65aef6a None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:52.770977 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44d71a-1348-4419-af5d-0156c65aef6a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:52.771250 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44d71a-1348-4419-af5d-0156c65aef6a None admin] PATH_INFO: `/v3/users/ff7198f9399a45d9bca9609255e07e6c` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:52.772389 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f44d71a-1348-4419-af5d-0156c65aef6a None admin] RBAC: Authorizing `identity:get_user(user_id=ff7198f9399a45d9bca9609255e07e6c)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:52.773811 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f44d71a-1348-4419-af5d-0156c65aef6a None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:52.775432 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 598/1190] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:10:52 2022] GET /identity/v3/users/ff7198f9399a45d9bca9609255e07e6c => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:52.781623 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14dd572-8c9f-4442-945d-07bc5c1f0119 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:52.882364 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14dd572-8c9f-4442-945d-07bc5c1f0119 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:52.915479 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8631f9a4-f2b0-4580-b9d3-7fbcbb1b05f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:52.916115 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8631f9a4-f2b0-4580-b9d3-7fbcbb1b05f5 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:52.916399 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8631f9a4-f2b0-4580-b9d3-7fbcbb1b05f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:52.916677 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8631f9a4-f2b0-4580-b9d3-7fbcbb1b05f5 None admin] PATH_INFO: `/v3/projects/91ca556673604d438fcf0945ec430937` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:52.917818 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8631f9a4-f2b0-4580-b9d3-7fbcbb1b05f5 None admin] RBAC: Authorizing `identity:get_project(project_id=91ca556673604d438fcf0945ec430937)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:52.919268 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8631f9a4-f2b0-4580-b9d3-7fbcbb1b05f5 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:52.922056 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 593/1191] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:10:52 2022] GET /identity/v3/projects/91ca556673604d438fcf0945ec430937 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:52.927915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f44d71a-1348-4419-af5d-0156c65aef6a None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:53.021266 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:53.024151 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f44d71a-1348-4419-af5d-0156c65aef6a None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:53.057317 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f9b21-a416-42e9-9311-61a13da5c88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:53.058002 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f9b21-a416-42e9-9311-61a13da5c88d None admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:53.058289 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f9b21-a416-42e9-9311-61a13da5c88d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:53.058567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f9b21-a416-42e9-9311-61a13da5c88d None admin] PATH_INFO: `/v3/projects/91ca556673604d438fcf0945ec430937/users/ff7198f9399a45d9bca9609255e07e6c/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:53.060478 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3f9b21-a416-42e9-9311-61a13da5c88d None admin] RBAC: Authorizing `identity:create_grant(project_id=91ca556673604d438fcf0945ec430937, user_id=ff7198f9399a45d9bca9609255e07e6c, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:53.062547 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3f9b21-a416-42e9-9311-61a13da5c88d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:53.074997 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 599/1192] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 27 04:10:52 2022] PUT /identity/v3/projects/91ca556673604d438fcf0945ec430937/users/ff7198f9399a45d9bca9609255e07e6c/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:10:55.042407 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b976d6f-ef3c-450b-9f15-7ac3b34f04e9 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:55.042957 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b976d6f-ef3c-450b-9f15-7ac3b34f04e9 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:55.042957 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b976d6f-ef3c-450b-9f15-7ac3b34f04e9 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:55.044296 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 594/1193] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:10:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:10:55.049860 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cfe59c-7604-4e02-be1a-9a2c3409f4fc None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:55.050316 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cfe59c-7604-4e02-be1a-9a2c3409f4fc None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:55.050444 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cfe59c-7604-4e02-be1a-9a2c3409f4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:55.068699 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-58cfe59c-7604-4e02-be1a-9a2c3409f4fc None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:10:55.069870 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-58cfe59c-7604-4e02-be1a-9a2c3409f4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:10:55.135326 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 600/1194] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:10:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:10:55.157661 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b976d6f-ef3c-450b-9f15-7ac3b34f04e9 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:55.215512 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:55.257295 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b976d6f-ef3c-450b-9f15-7ac3b34f04e9 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:55.288540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2874b8d6-267d-41f9-bfaa-57f777f55167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:55.289164 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874b8d6-267d-41f9-bfaa-57f777f55167 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:55.289441 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874b8d6-267d-41f9-bfaa-57f777f55167 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:55.289711 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874b8d6-267d-41f9-bfaa-57f777f55167 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:55.300316 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2874b8d6-267d-41f9-bfaa-57f777f55167 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:10:55.305021 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-2874b8d6-267d-41f9-bfaa-57f777f55167 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:10:55.305021 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:10:55.305021 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:10:55.305021 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:10:55.305021 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:10:55.305021 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:10:55.305021 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:10:55.305021 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:10:55.305021 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:10:55.305021 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:10:55.305021 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:10:55.305021 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:10:55.305021 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:10:55.305021 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:10:55.305021 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:10:55.305021 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:10:55.306140 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:10:55.306140 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:10:55.306140 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:10:55.306140 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:10:55.306140 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:10:55.306140 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:10:55.306140 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:10:55.306140 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:10:55.306140 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:10:55.306140 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:10:55.306140 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:10:55.306140 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:10:55.306140 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:10:55.306140 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:10:55.306140 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:10:55.306140 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:10:55.307345 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:10:55.307345 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:10:55.307345 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:10:55.307345 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:10:55.307345 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:10:55.307345 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:10:55.307345 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:10:55.307345 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:10:55.307345 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:10:55.307345 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:10:55.307345 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:10:55.307345 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:10:55.307345 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 595/1195] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:10:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:10:55.312781 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cfe59c-7604-4e02-be1a-9a2c3409f4fc None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:55.398123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:55.409286 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cfe59c-7604-4e02-be1a-9a2c3409f4fc None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:55.443284 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c44e02-5d6d-431d-b3b0-c936d3e3e733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:55.443902 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c44e02-5d6d-431d-b3b0-c936d3e3e733 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:55.444187 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c44e02-5d6d-431d-b3b0-c936d3e3e733 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:55.444463 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c44e02-5d6d-431d-b3b0-c936d3e3e733 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:55.445193 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c44e02-5d6d-431d-b3b0-c936d3e3e733 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:55.445500 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c44e02-5d6d-431d-b3b0-c936d3e3e733 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:55.446898 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c44e02-5d6d-431d-b3b0-c936d3e3e733 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:55.454926 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 601/1196] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:10:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:55.460639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2874b8d6-267d-41f9-bfaa-57f777f55167 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:55.558372 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2874b8d6-267d-41f9-bfaa-57f777f55167 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:55.577836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:55.592631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff002e90-84f9-412d-8d6a-44e4b7a1adf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:55.593407 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff002e90-84f9-412d-8d6a-44e4b7a1adf2 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:55.593687 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff002e90-84f9-412d-8d6a-44e4b7a1adf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:55.593958 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff002e90-84f9-412d-8d6a-44e4b7a1adf2 None admin] PATH_INFO: `/v3/users/ff7198f9399a45d9bca9609255e07e6c` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:55.595144 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff002e90-84f9-412d-8d6a-44e4b7a1adf2 None admin] RBAC: Authorizing `identity:get_user(user_id=ff7198f9399a45d9bca9609255e07e6c)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:55.596586 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff002e90-84f9-412d-8d6a-44e4b7a1adf2 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:55.598208 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 596/1197] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 27 04:10:55 2022] GET /identity/v3/users/ff7198f9399a45d9bca9609255e07e6c => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:55.604093 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c44e02-5d6d-431d-b3b0-c936d3e3e733 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:55.702512 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c44e02-5d6d-431d-b3b0-c936d3e3e733 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:55.736308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449cdbd3-a835-4918-99b6-763db49d6e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:55.736921 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cdbd3-a835-4918-99b6-763db49d6e45 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:55.737204 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cdbd3-a835-4918-99b6-763db49d6e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:55.737482 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cdbd3-a835-4918-99b6-763db49d6e45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:55.758568 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:55.803090 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449cdbd3-a835-4918-99b6-763db49d6e45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:55.804530 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449cdbd3-a835-4918-99b6-763db49d6e45 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:55.899594 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 602/1198] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:10:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 04:10:55.905408 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff002e90-84f9-412d-8d6a-44e4b7a1adf2 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:56.006771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff002e90-84f9-412d-8d6a-44e4b7a1adf2 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:56.040300 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a935b11c-ae8e-4363-a5df-0fdb5ff19faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:56.040935 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935b11c-ae8e-4363-a5df-0fdb5ff19faf None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:56.041216 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935b11c-ae8e-4363-a5df-0fdb5ff19faf None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:56.041490 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935b11c-ae8e-4363-a5df-0fdb5ff19faf None admin] PATH_INFO: `/v3/projects/91ca556673604d438fcf0945ec430937` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:56.042646 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a935b11c-ae8e-4363-a5df-0fdb5ff19faf None admin] RBAC: Authorizing `identity:get_project(project_id=91ca556673604d438fcf0945ec430937)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:56.044079 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a935b11c-ae8e-4363-a5df-0fdb5ff19faf None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:56.046319 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 597/1199] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 27 04:10:55 2022] GET /identity/v3/projects/91ca556673604d438fcf0945ec430937 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:10:56.052292 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449cdbd3-a835-4918-99b6-763db49d6e45 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:10:56.103328 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:10:56.149247 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449cdbd3-a835-4918-99b6-763db49d6e45 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:10:56.181567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92c0a2-a419-4c20-8f10-a12a42337178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:10:56.182148 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92c0a2-a419-4c20-8f10-a12a42337178 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:10:56.182434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92c0a2-a419-4c20-8f10-a12a42337178 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:10:56.182711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92c0a2-a419-4c20-8f10-a12a42337178 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:10:56.183502 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92c0a2-a419-4c20-8f10-a12a42337178 None admin] RBAC: Adding query filter params (role.id=cf771a8e94ee46149eb081347fea6d67, scope.project.id=91ca556673604d438fcf0945ec430937, user.id=ff7198f9399a45d9bca9609255e07e6c) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:10:56.183841 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92c0a2-a419-4c20-8f10-a12a42337178 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:10:56.185304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92c0a2-a419-4c20-8f10-a12a42337178 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:10:56.191832 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 603/1200] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:10:56 2022] GET /identity/v3/role_assignments?user.id=ff7198f9399a45d9bca9609255e07e6c&scope.project.id=91ca556673604d438fcf0945ec430937&role.id=cf771a8e94ee46149eb081347fea6d67 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:11:06.036324 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=72 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:11:06.189373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=42 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:11:51.432616 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704015a6-c848-4e44-885f-b6f5f50746ca None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:11:51.433290 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704015a6-c848-4e44-885f-b6f5f50746ca None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:11:51.433290 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704015a6-c848-4e44-885f-b6f5f50746ca None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:11:51.436001 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 598/1201] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:11:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:11:51.443125 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85c01a-4dd6-459b-88e7-d7ed3ade80e6 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:11:51.443613 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85c01a-4dd6-459b-88e7-d7ed3ade80e6 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:11:51.443824 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85c01a-4dd6-459b-88e7-d7ed3ade80e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:11:51.463943 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-bd85c01a-4dd6-459b-88e7-d7ed3ade80e6 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:11:51.466014 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-bd85c01a-4dd6-459b-88e7-d7ed3ade80e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:11:51.558965 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 604/1202] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:11:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:11:51.583843 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704015a6-c848-4e44-885f-b6f5f50746ca None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:11:51.723817 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704015a6-c848-4e44-885f-b6f5f50746ca None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:11:51.782000 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a0fa2-57d9-4dc4-807a-2e378fb5577f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:11:51.782647 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a0fa2-57d9-4dc4-807a-2e378fb5577f None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:11:51.782914 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a0fa2-57d9-4dc4-807a-2e378fb5577f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:11:51.783318 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a0fa2-57d9-4dc4-807a-2e378fb5577f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:11:51.797041 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a0fa2-57d9-4dc4-807a-2e378fb5577f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:11:51.798889 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a0fa2-57d9-4dc4-807a-2e378fb5577f None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:11:51.815990 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-1e2a0fa2-57d9-4dc4-807a-2e378fb5577f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:11:51.815990 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:11:51.815990 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:11:51.815990 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:11:51.815990 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:11:51.815990 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:11:51.815990 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:11:51.815990 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:11:51.815990 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:11:51.815990 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:11:51.815990 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:11:51.815990 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:11:51.815990 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:11:51.815990 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:11:51.815990 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:11:51.817099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:11:51.817099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:11:51.817099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:11:51.817099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:11:51.817099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:11:51.817099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:11:51.817099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:11:51.817099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:11:51.817099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:11:51.817099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:11:51.817099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:11:51.817099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:11:51.817099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:11:51.817099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:11:51.817099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:11:51.817099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:11:51.817099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:11:51.823674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:11:51.823674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:11:51.823674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:11:51.823674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:11:51.823674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 599/1203] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:11:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:11:51.824713 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85c01a-4dd6-459b-88e7-d7ed3ade80e6 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:11:51.963314 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85c01a-4dd6-459b-88e7-d7ed3ade80e6 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:11:52.002496 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:11:52.010239 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f020225e-cafc-442e-9997-965a21181318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:11:52.010959 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020225e-cafc-442e-9997-965a21181318 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:11:52.011925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020225e-cafc-442e-9997-965a21181318 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:11:52.012578 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020225e-cafc-442e-9997-965a21181318 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:11:52.013468 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f020225e-cafc-442e-9997-965a21181318 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:11:52.013901 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f020225e-cafc-442e-9997-965a21181318 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:11:52.015726 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f020225e-cafc-442e-9997-965a21181318 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:11:52.039408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 605/1204] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:11:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:11:52.041696 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a0fa2-57d9-4dc4-807a-2e378fb5577f None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:11:52.118954 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:11:52.177616 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a0fa2-57d9-4dc4-807a-2e378fb5577f None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:11:52.236224 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aeeb3a-a82d-42ba-85fb-80e91139ad77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:11:52.237171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aeeb3a-a82d-42ba-85fb-80e91139ad77 None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:11:52.237535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aeeb3a-a82d-42ba-85fb-80e91139ad77 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:11:52.237876 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aeeb3a-a82d-42ba-85fb-80e91139ad77 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:11:52.238439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aeeb3a-a82d-42ba-85fb-80e91139ad77 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:11:52.239898 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aeeb3a-a82d-42ba-85fb-80e91139ad77 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:11:52.270570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 600/1205] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 27 04:11:52 2022] POST /identity/v3/users => generated 251 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:11:54.513869 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2013fc-015f-47ec-be69-04f3649fbcae None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:11:54.514461 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2013fc-015f-47ec-be69-04f3649fbcae None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:11:54.514461 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2013fc-015f-47ec-be69-04f3649fbcae None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:11:54.515958 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 606/1206] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:11:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:11:54.521638 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a5493-2b20-4443-8c63-09ffa365dfbe None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:11:54.522132 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a5493-2b20-4443-8c63-09ffa365dfbe None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:11:54.522247 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a5493-2b20-4443-8c63-09ffa365dfbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:11:54.542474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-d24a5493-2b20-4443-8c63-09ffa365dfbe None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:11:54.543758 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d24a5493-2b20-4443-8c63-09ffa365dfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:11:54.623103 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 601/1207] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:11:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:11:54.646306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2013fc-015f-47ec-be69-04f3649fbcae None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:11:54.767053 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2013fc-015f-47ec-be69-04f3649fbcae None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:11:54.814946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b3fe4-2e94-459e-b6cf-1f72ccc6b8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:11:54.815641 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b3fe4-2e94-459e-b6cf-1f72ccc6b8b8 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:11:54.815901 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b3fe4-2e94-459e-b6cf-1f72ccc6b8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:11:54.816176 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b3fe4-2e94-459e-b6cf-1f72ccc6b8b8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:11:54.829112 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b3fe4-2e94-459e-b6cf-1f72ccc6b8b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:11:54.833794 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-723b3fe4-2e94-459e-b6cf-1f72ccc6b8b8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:11:54.833794 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:11:54.833794 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:11:54.833794 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:11:54.833794 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:11:54.833794 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:11:54.833794 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:11:54.833794 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:11:54.833794 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:11:54.833794 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:11:54.833794 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:11:54.833794 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:11:54.833794 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:11:54.833794 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:11:54.833794 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:11:54.833794 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:11:54.834830 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:11:54.834830 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:11:54.834830 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:11:54.834830 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:11:54.834830 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:11:54.834830 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:11:54.834830 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:11:54.834830 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:11:54.834830 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:11:54.834830 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:11:54.834830 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:11:54.834830 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:11:54.834830 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:11:54.834830 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:11:54.834830 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:11:54.834830 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:11:54.835695 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:11:54.835695 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:11:54.835695 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:11:54.835695 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:11:54.835695 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:11:54.835695 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:11:54.835695 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:11:54.835695 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:11:54.835695 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:11:54.835695 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:11:54.835695 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:11:54.835695 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:11:54.835695 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 607/1208] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 27 04:11:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:11:54.841926 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a5493-2b20-4443-8c63-09ffa365dfbe None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:11:54.887775 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:11:54.959682 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a5493-2b20-4443-8c63-09ffa365dfbe None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:11:54.998975 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2aae25-44a7-41ce-b200-2c16ea689775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:11:54.999654 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2aae25-44a7-41ce-b200-2c16ea689775 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:11:54.999962 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2aae25-44a7-41ce-b200-2c16ea689775 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:11:55.000294 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2aae25-44a7-41ce-b200-2c16ea689775 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:11:55.001171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2aae25-44a7-41ce-b200-2c16ea689775 None admin] RBAC: Adding query filter params (name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:11:55.001452 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2aae25-44a7-41ce-b200-2c16ea689775 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:11:55.002897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2aae25-44a7-41ce-b200-2c16ea689775 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:11:55.012058 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 602/1209] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 27 04:11:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:11:55.019734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b3fe4-2e94-459e-b6cf-1f72ccc6b8b8 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:11:55.089059 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:11:55.158170 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b3fe4-2e94-459e-b6cf-1f72ccc6b8b8 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:11:55.198053 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6808c1-e31a-4ead-881d-dcff8925bf21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:11:55.198817 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6808c1-e31a-4ead-881d-dcff8925bf21 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:11:55.199122 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6808c1-e31a-4ead-881d-dcff8925bf21 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:11:55.199398 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6808c1-e31a-4ead-881d-dcff8925bf21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:11:55.214470 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6808c1-e31a-4ead-881d-dcff8925bf21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:11:55.216890 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6808c1-e31a-4ead-881d-dcff8925bf21 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:11:55.235509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-5b6808c1-e31a-4ead-881d-dcff8925bf21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:11:55.235509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:11:55.235509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:11:55.235509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:11:55.235509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:11:55.235509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:11:55.235509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:11:55.235509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:11:55.235509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:11:55.235509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:11:55.235509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:11:55.235509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:11:55.235509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:11:55.235509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:11:55.235509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:11:55.236863 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:11:55.236863 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:11:55.236863 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:11:55.236863 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:11:55.236863 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:11:55.236863 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:11:55.236863 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:11:55.236863 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:11:55.236863 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:11:55.236863 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:11:55.236863 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:11:55.236863 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:11:55.236863 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:11:55.236863 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:11:55.236863 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:11:55.236863 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:11:55.236863 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:11:55.238241 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:11:55.238241 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:11:55.238241 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:11:55.238241 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:11:55.238241 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 608/1210] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:11:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:11:55.243676 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2aae25-44a7-41ce-b200-2c16ea689775 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:11:55.361270 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:11:55.365423 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2aae25-44a7-41ce-b200-2c16ea689775 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:11:55.407792 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb30ea0-4701-4285-80f7-0ade72c9be97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:11:55.408507 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb30ea0-4701-4285-80f7-0ade72c9be97 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:11:55.408857 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb30ea0-4701-4285-80f7-0ade72c9be97 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:11:55.409143 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb30ea0-4701-4285-80f7-0ade72c9be97 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:11:55.410079 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb30ea0-4701-4285-80f7-0ade72c9be97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:11:55.410394 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb30ea0-4701-4285-80f7-0ade72c9be97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:11:55.411936 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb30ea0-4701-4285-80f7-0ade72c9be97 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:11:55.428661 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 603/1211] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:11:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:11:55.432750 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6808c1-e31a-4ead-881d-dcff8925bf21 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:11:55.555077 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6808c1-e31a-4ead-881d-dcff8925bf21 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:11:55.556765 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:11:55.598029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e76747-f0c5-4c3d-af9f-4f23ec7d6919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:11:55.599253 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e76747-f0c5-4c3d-af9f-4f23ec7d6919 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:11:55.599899 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e76747-f0c5-4c3d-af9f-4f23ec7d6919 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:11:55.600388 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e76747-f0c5-4c3d-af9f-4f23ec7d6919 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:11:55.610025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e76747-f0c5-4c3d-af9f-4f23ec7d6919 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:11:55.612013 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e76747-f0c5-4c3d-af9f-4f23ec7d6919 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:11:55.626598 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-00e76747-f0c5-4c3d-af9f-4f23ec7d6919 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 04:11:55.626598 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:11:55.626598 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:11:55.626598 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:11:55.626598 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:11:55.626598 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:11:55.626598 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:11:55.626598 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:11:55.626598 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:11:55.626598 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:11:55.626598 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:11:55.626598 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:11:55.626598 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:11:55.626598 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:11:55.626598 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:11:55.626598 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:11:55.627689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:11:55.627689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:11:55.627689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:11:55.627689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:11:55.627689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:11:55.627689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:11:55.627689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:11:55.627689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:11:55.627689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:11:55.627689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:11:55.627689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:11:55.627689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:11:55.627689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:11:55.627689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:11:55.627689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:11:55.627689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:11:55.627689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:11:55.627689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:11:55.628666 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:11:55.628666 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:11:55.628666 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:11:55.628666 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:11:55.628666 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:11:55.628666 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:11:55.628666 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:11:55.628666 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:11:55.628666 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 04:11:55.628666 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:11:55.628666 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 609/1212] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:11:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:11:55.634579 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb30ea0-4701-4285-80f7-0ade72c9be97 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:11:55.753045 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb30ea0-4701-4285-80f7-0ade72c9be97 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:11:55.792664 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239beec-75ca-4366-bf9e-6f7328c4d38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:11:55.793327 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239beec-75ca-4366-bf9e-6f7328c4d38b None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:11:55.794208 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239beec-75ca-4366-bf9e-6f7328c4d38b None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:11:55.794617 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239beec-75ca-4366-bf9e-6f7328c4d38b None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:11:55.795512 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239beec-75ca-4366-bf9e-6f7328c4d38b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:11:55.795933 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239beec-75ca-4366-bf9e-6f7328c4d38b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:11:55.797384 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239beec-75ca-4366-bf9e-6f7328c4d38b None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:11:55.819425 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 604/1213] 10.222.0.46 () {56 vars in 1143 bytes} [Wed Jul 27 04:11:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:11:55.826048 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e76747-f0c5-4c3d-af9f-4f23ec7d6919 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:11:55.947198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e76747-f0c5-4c3d-af9f-4f23ec7d6919 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:11:55.985208 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e538abce-8754-45c0-8643-2aa51263924d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:11:55.986721 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538abce-8754-45c0-8643-2aa51263924d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:11:55.986721 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538abce-8754-45c0-8643-2aa51263924d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:11:55.986721 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538abce-8754-45c0-8643-2aa51263924d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:11:56.025471 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:11:56.068537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e538abce-8754-45c0-8643-2aa51263924d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:11:56.070213 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e538abce-8754-45c0-8643-2aa51263924d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:11:56.201829 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 610/1214] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:11:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 04:11:56.207552 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239beec-75ca-4366-bf9e-6f7328c4d38b None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:11:56.233573 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:11:56.334650 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239beec-75ca-4366-bf9e-6f7328c4d38b None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:11:56.381041 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc6afa-9fe6-4a87-b4ad-e58f39a26df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:11:56.381874 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc6afa-9fe6-4a87-b4ad-e58f39a26df2 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:11:56.382316 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc6afa-9fe6-4a87-b4ad-e58f39a26df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:11:56.382738 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc6afa-9fe6-4a87-b4ad-e58f39a26df2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:11:56.396049 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcc6afa-9fe6-4a87-b4ad-e58f39a26df2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:11:56.397657 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcc6afa-9fe6-4a87-b4ad-e58f39a26df2 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:11:56.413535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-ddcc6afa-9fe6-4a87-b4ad-e58f39a26df2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:11:56.413535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:11:56.413535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:11:56.413535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:11:56.413535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:11:56.413535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:11:56.413535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:11:56.413535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:11:56.413535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:11:56.413535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:11:56.413535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:11:56.413535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:11:56.413535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:11:56.413535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:11:56.413535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:11:56.414779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:11:56.414779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:11:56.414779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:11:56.414779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:11:56.414779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:11:56.414779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:11:56.414779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:11:56.414779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:11:56.414779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:11:56.414779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:11:56.414779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:11:56.414779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:11:56.414779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:11:56.414779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:11:56.414779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:11:56.414779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:11:56.414779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:11:56.415911 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:11:56.415911 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:11:56.415911 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:11:56.415911 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:11:56.416605 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 605/1215] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:11:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:11:56.422898 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e538abce-8754-45c0-8643-2aa51263924d None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:11:56.502224 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:11:56.550730 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e538abce-8754-45c0-8643-2aa51263924d None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:11:56.594163 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c55ae-886a-4a6e-be08-0820d28a41b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:11:56.594812 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c55ae-886a-4a6e-be08-0820d28a41b8 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:11:56.595077 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c55ae-886a-4a6e-be08-0820d28a41b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:11:56.595348 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c55ae-886a-4a6e-be08-0820d28a41b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:11:56.596062 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6c55ae-886a-4a6e-be08-0820d28a41b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:11:56.596365 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6c55ae-886a-4a6e-be08-0820d28a41b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:11:56.597826 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6c55ae-886a-4a6e-be08-0820d28a41b8 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:11:56.618054 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 611/1216] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:11:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:11:56.620756 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc6afa-9fe6-4a87-b4ad-e58f39a26df2 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:11:56.702272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:11:56.738130 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc6afa-9fe6-4a87-b4ad-e58f39a26df2 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:11:56.778972 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e4848-cfb3-443a-888e-12385dbc8fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:11:56.779684 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e4848-cfb3-443a-888e-12385dbc8fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:11:56.779962 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e4848-cfb3-443a-888e-12385dbc8fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:11:56.780228 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e4848-cfb3-443a-888e-12385dbc8fd8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:11:56.787259 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578e4848-cfb3-443a-888e-12385dbc8fd8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:11:56.790267 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578e4848-cfb3-443a-888e-12385dbc8fd8 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:11:56.802937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-578e4848-cfb3-443a-888e-12385dbc8fd8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:11:56.802937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:11:56.802937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:11:56.802937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:11:56.802937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:11:56.802937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:11:56.802937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:11:56.802937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:11:56.802937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:11:56.802937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:11:56.802937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:11:56.802937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:11:56.802937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:11:56.802937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:11:56.802937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:11:56.805555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:11:56.805555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:11:56.805555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:11:56.805555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:11:56.805555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:11:56.805555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:11:56.805555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:11:56.805555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:11:56.805555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:11:56.805555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:11:56.805555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:11:56.805555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:11:56.805555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:11:56.805555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:11:56.805555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:11:56.805555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:11:56.805555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:11:56.806685 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:11:56.806685 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:11:56.806685 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:11:56.806685 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:11:56.806685 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:11:56.806685 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:11:56.806685 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:11:56.806685 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:11:56.806685 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 606/1217] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:11:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:11:56.810539 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c55ae-886a-4a6e-be08-0820d28a41b8 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:11:56.921392 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c55ae-886a-4a6e-be08-0820d28a41b8 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:11:56.952882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:11:56.968546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e8b84-2e18-4d50-a4f7-f9ff81e0e228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:11:56.969414 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e8b84-2e18-4d50-a4f7-f9ff81e0e228 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:11:56.969809 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e8b84-2e18-4d50-a4f7-f9ff81e0e228 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:11:56.970179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e8b84-2e18-4d50-a4f7-f9ff81e0e228 None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:11:56.970948 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7e8b84-2e18-4d50-a4f7-f9ff81e0e228 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:11:56.971262 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7e8b84-2e18-4d50-a4f7-f9ff81e0e228 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:11:56.972697 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7e8b84-2e18-4d50-a4f7-f9ff81e0e228 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:11:56.987360 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 612/1218] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:11:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:11:56.993598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e4848-cfb3-443a-888e-12385dbc8fd8 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:11:57.117409 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e4848-cfb3-443a-888e-12385dbc8fd8 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:11:57.151131 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:11:57.158209 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dba6f9-6e46-4549-9546-a7d88a86d430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:11:57.158927 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dba6f9-6e46-4549-9546-a7d88a86d430 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:11:57.159274 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dba6f9-6e46-4549-9546-a7d88a86d430 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:11:57.159614 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dba6f9-6e46-4549-9546-a7d88a86d430 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:11:57.160477 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dba6f9-6e46-4549-9546-a7d88a86d430 None admin] RBAC: Adding query filter params (role.id=6521132834a64b029695bbdd5eb251d2, scope.project.id=e646d320aa5a4f78bfccaf941cf46478, user.id=8c08fba2d7de42218400c5e652cc3de0) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:11:57.160853 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dba6f9-6e46-4549-9546-a7d88a86d430 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:11:57.162423 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dba6f9-6e46-4549-9546-a7d88a86d430 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:11:57.169560 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 607/1219] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:11:56 2022] GET /identity/v3/role_assignments?user.id=8c08fba2d7de42218400c5e652cc3de0&scope.project.id=e646d320aa5a4f78bfccaf941cf46478&role.id=6521132834a64b029695bbdd5eb251d2 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:11:59.257730 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd257abc-dc06-483e-a907-fb91bb774dc4 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:11:59.258344 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd257abc-dc06-483e-a907-fb91bb774dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:11:59.258344 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd257abc-dc06-483e-a907-fb91bb774dc4 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:11:59.260107 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 613/1220] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:11:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 04:11:59.265337 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5f51d-1de7-4a7d-9271-1b6b7ea0a7c2 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:11:59.265826 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5f51d-1de7-4a7d-9271-1b6b7ea0a7c2 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:11:59.265964 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5f51d-1de7-4a7d-9271-1b6b7ea0a7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:11:59.286511 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-dca5f51d-1de7-4a7d-9271-1b6b7ea0a7c2 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:11:59.287726 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-dca5f51d-1de7-4a7d-9271-1b6b7ea0a7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:11:59.359099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 608/1221] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:11:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:11:59.381675 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd257abc-dc06-483e-a907-fb91bb774dc4 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:11:59.495407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd257abc-dc06-483e-a907-fb91bb774dc4 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:11:59.532491 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b5d09-c474-4357-a321-7ce060734cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:11:59.533099 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b5d09-c474-4357-a321-7ce060734cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:11:59.533369 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b5d09-c474-4357-a321-7ce060734cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:11:59.533669 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b5d09-c474-4357-a321-7ce060734cb0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:11:59.544414 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b5d09-c474-4357-a321-7ce060734cb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:11:59.549524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-cd4b5d09-c474-4357-a321-7ce060734cb0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:11:59.549524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:11:59.549524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:11:59.549524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:11:59.549524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:11:59.549524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:11:59.549524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:11:59.549524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:11:59.549524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:11:59.549524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:11:59.549524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:11:59.549524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:11:59.549524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:11:59.549524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:11:59.549524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:11:59.549524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:11:59.550555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:11:59.550555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:11:59.550555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:11:59.550555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:11:59.550555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:11:59.550555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:11:59.550555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:11:59.550555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:11:59.550555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:11:59.550555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:11:59.550555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:11:59.550555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:11:59.550555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:11:59.550555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:11:59.550555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:11:59.550555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:11:59.551396 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:11:59.551396 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:11:59.551396 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:11:59.551396 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:11:59.551396 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:11:59.551396 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:11:59.551396 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:11:59.551396 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:11:59.551396 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:11:59.551396 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:11:59.551396 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:11:59.551396 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:11:59.551396 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 614/1222] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 27 04:11:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:11:59.557217 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5f51d-1de7-4a7d-9271-1b6b7ea0a7c2 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:11:59.659152 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5f51d-1de7-4a7d-9271-1b6b7ea0a7c2 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:11:59.682777 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:11:59.695668 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d2502-1333-4ef3-87b9-5759ff224833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:11:59.696377 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d2502-1333-4ef3-87b9-5759ff224833 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:11:59.696683 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d2502-1333-4ef3-87b9-5759ff224833 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:11:59.697023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d2502-1333-4ef3-87b9-5759ff224833 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:11:59.697763 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d2502-1333-4ef3-87b9-5759ff224833 None admin] RBAC: Adding query filter params (name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:11:59.698158 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d2502-1333-4ef3-87b9-5759ff224833 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:11:59.699635 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d2502-1333-4ef3-87b9-5759ff224833 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:11:59.708159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 609/1223] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 27 04:11:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:11:59.714219 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b5d09-c474-4357-a321-7ce060734cb0 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:11:59.761158 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:11:59.827282 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b5d09-c474-4357-a321-7ce060734cb0 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:11:59.862310 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a35970-641b-40f6-9542-a28716bb7865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:11:59.862951 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a35970-641b-40f6-9542-a28716bb7865 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:11:59.863232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a35970-641b-40f6-9542-a28716bb7865 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:11:59.863504 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a35970-641b-40f6-9542-a28716bb7865 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:11:59.873876 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a35970-641b-40f6-9542-a28716bb7865 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:11:59.875335 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a35970-641b-40f6-9542-a28716bb7865 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:11:59.890329 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-15a35970-641b-40f6-9542-a28716bb7865 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:11:59.890329 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:11:59.890329 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:11:59.890329 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:11:59.890329 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:11:59.890329 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:11:59.890329 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:11:59.890329 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:11:59.890329 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:11:59.890329 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:11:59.890329 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:11:59.890329 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:11:59.890329 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:11:59.890329 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:11:59.890329 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:11:59.891418 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:11:59.891418 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:11:59.891418 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:11:59.891418 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:11:59.891418 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:11:59.891418 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:11:59.891418 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:11:59.891418 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:11:59.891418 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:11:59.891418 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:11:59.891418 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:11:59.891418 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:11:59.891418 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:11:59.891418 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:11:59.891418 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:11:59.891418 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:11:59.891418 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:11:59.892344 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:11:59.892344 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:11:59.892344 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:11:59.892344 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:11:59.892344 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 615/1224] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:11:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:11:59.897360 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d2502-1333-4ef3-87b9-5759ff224833 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:00.007440 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d2502-1333-4ef3-87b9-5759ff224833 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:00.042163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae2c9d-f87e-4699-a70f-dd6f5307eb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:00.042796 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae2c9d-f87e-4699-a70f-dd6f5307eb04 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:00.043082 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae2c9d-f87e-4699-a70f-dd6f5307eb04 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:00.043553 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae2c9d-f87e-4699-a70f-dd6f5307eb04 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:00.044341 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae2c9d-f87e-4699-a70f-dd6f5307eb04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:12:00.044682 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae2c9d-f87e-4699-a70f-dd6f5307eb04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:00.046613 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae2c9d-f87e-4699-a70f-dd6f5307eb04 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:00.060186 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 610/1225] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:11:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:12:00.066060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a35970-641b-40f6-9542-a28716bb7865 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:00.170757 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:00.179992 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a35970-641b-40f6-9542-a28716bb7865 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:00.223190 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c05311-db3e-4264-8c65-d74c6b1e2484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:00.223955 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c05311-db3e-4264-8c65-d74c6b1e2484 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:00.224242 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c05311-db3e-4264-8c65-d74c6b1e2484 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:00.224645 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c05311-db3e-4264-8c65-d74c6b1e2484 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:00.234567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c05311-db3e-4264-8c65-d74c6b1e2484 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:00.236238 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c05311-db3e-4264-8c65-d74c6b1e2484 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:00.249676 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-87c05311-db3e-4264-8c65-d74c6b1e2484 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 04:12:00.249676 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:12:00.249676 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:12:00.249676 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:12:00.249676 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:12:00.249676 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:12:00.249676 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:12:00.249676 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:12:00.249676 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:12:00.249676 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:12:00.249676 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:12:00.249676 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:12:00.249676 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:12:00.249676 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:12:00.249676 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:12:00.249676 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:12:00.250798 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:12:00.250798 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:12:00.250798 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:12:00.250798 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:12:00.250798 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:12:00.250798 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:12:00.250798 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:12:00.250798 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:12:00.250798 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:12:00.250798 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:12:00.250798 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:12:00.250798 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:12:00.250798 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:12:00.250798 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:12:00.250798 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:12:00.250798 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:12:00.250798 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:12:00.250798 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:12:00.251776 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:12:00.251776 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:12:00.251776 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:12:00.251776 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:12:00.251776 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:12:00.251776 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:12:00.251776 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:12:00.251776 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:12:00.251776 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 04:12:00.251776 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:12:00.251776 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 616/1226] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:12:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:12:00.257204 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae2c9d-f87e-4699-a70f-dd6f5307eb04 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:00.312816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:00.371125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae2c9d-f87e-4699-a70f-dd6f5307eb04 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:00.411763 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328cec00-4c55-4c69-89fe-70fad16a812d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:00.412391 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cec00-4c55-4c69-89fe-70fad16a812d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:00.412665 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cec00-4c55-4c69-89fe-70fad16a812d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:00.412934 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cec00-4c55-4c69-89fe-70fad16a812d None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:00.413731 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328cec00-4c55-4c69-89fe-70fad16a812d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:12:00.414175 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328cec00-4c55-4c69-89fe-70fad16a812d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:00.415754 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328cec00-4c55-4c69-89fe-70fad16a812d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:00.435457 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 611/1227] 10.222.0.46 () {56 vars in 1143 bytes} [Wed Jul 27 04:12:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:12:00.441491 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c05311-db3e-4264-8c65-d74c6b1e2484 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:00.558345 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c05311-db3e-4264-8c65-d74c6b1e2484 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:00.596481 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037bb59-e9d7-414b-8cc6-fc49b547273e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:00.597233 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037bb59-e9d7-414b-8cc6-fc49b547273e None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:00.597580 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037bb59-e9d7-414b-8cc6-fc49b547273e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:00.597859 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037bb59-e9d7-414b-8cc6-fc49b547273e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:00.609225 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8037bb59-e9d7-414b-8cc6-fc49b547273e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:00.610723 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8037bb59-e9d7-414b-8cc6-fc49b547273e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:00.630573 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-8037bb59-e9d7-414b-8cc6-fc49b547273e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:12:00.630573 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:12:00.630573 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:12:00.630573 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:12:00.630573 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:12:00.630573 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:12:00.630573 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:12:00.630573 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:12:00.630573 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:12:00.630573 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:12:00.630573 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:12:00.630573 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:12:00.630573 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:12:00.630573 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:12:00.630573 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:12:00.631730 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:12:00.631730 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:12:00.631730 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:12:00.631730 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:12:00.631730 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:12:00.631730 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:12:00.631730 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:12:00.631730 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:12:00.631730 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:12:00.631730 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:12:00.631730 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:12:00.631730 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:12:00.631730 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:12:00.631730 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:12:00.631730 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:12:00.631730 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:12:00.631730 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:12:00.632862 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:12:00.632862 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:12:00.632862 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:12:00.632862 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:12:00.632862 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 617/1228] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:12:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:12:00.638659 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328cec00-4c55-4c69-89fe-70fad16a812d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:00.750897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328cec00-4c55-4c69-89fe-70fad16a812d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:00.753548 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:00.791037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e754b9-2a18-4963-95fe-0b818a4ae612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:00.791689 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e754b9-2a18-4963-95fe-0b818a4ae612 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:00.791966 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e754b9-2a18-4963-95fe-0b818a4ae612 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:00.792241 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e754b9-2a18-4963-95fe-0b818a4ae612 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:00.792942 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e754b9-2a18-4963-95fe-0b818a4ae612 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:12:00.793269 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e754b9-2a18-4963-95fe-0b818a4ae612 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:00.795125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e754b9-2a18-4963-95fe-0b818a4ae612 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:00.806836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 612/1229] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:12:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:12:00.814753 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037bb59-e9d7-414b-8cc6-fc49b547273e None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:00.823971 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:00.922719 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037bb59-e9d7-414b-8cc6-fc49b547273e None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:00.959150 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f668c-7338-4104-953d-07a8a771ecb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:00.959793 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f668c-7338-4104-953d-07a8a771ecb4 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:00.960071 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f668c-7338-4104-953d-07a8a771ecb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:00.960342 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f668c-7338-4104-953d-07a8a771ecb4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:00.966973 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f668c-7338-4104-953d-07a8a771ecb4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:00.968414 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f668c-7338-4104-953d-07a8a771ecb4 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:00.980340 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-7c9f668c-7338-4104-953d-07a8a771ecb4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:12:00.980340 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:12:00.980340 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:12:00.980340 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:12:00.980340 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:12:00.980340 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:12:00.980340 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:12:00.980340 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:12:00.980340 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:12:00.980340 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:12:00.980340 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:12:00.980340 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:12:00.980340 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:12:00.980340 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:12:00.980340 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:12:00.982180 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:12:00.982180 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:12:00.982180 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:12:00.982180 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:12:00.982180 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:12:00.982180 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:12:00.982180 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:12:00.982180 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:12:00.982180 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:12:00.982180 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:12:00.982180 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:12:00.982180 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:12:00.982180 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:12:00.982180 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:12:00.982180 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:12:00.982180 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:12:00.982180 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:12:00.983626 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:12:00.983626 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:12:00.983626 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:12:00.983626 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:12:00.983626 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:12:00.983626 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:12:00.983626 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:12:00.983626 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:12:00.983626 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 618/1230] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:12:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 04:12:00.988654 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e754b9-2a18-4963-95fe-0b818a4ae612 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:01.104442 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e754b9-2a18-4963-95fe-0b818a4ae612 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:01.142242 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6510de-6523-4b50-b684-9832d3f5b8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:01.142879 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6510de-6523-4b50-b684-9832d3f5b8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:01.143210 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6510de-6523-4b50-b684-9832d3f5b8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:01.143500 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6510de-6523-4b50-b684-9832d3f5b8f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:01.144302 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6510de-6523-4b50-b684-9832d3f5b8f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:12:01.144610 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6510de-6523-4b50-b684-9832d3f5b8f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:01.146070 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6510de-6523-4b50-b684-9832d3f5b8f7 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:01.159041 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 613/1231] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:12:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:12:01.165453 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f668c-7338-4104-953d-07a8a771ecb4 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:01.241147 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:01.281750 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f668c-7338-4104-953d-07a8a771ecb4 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:01.318654 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835919e-d5ff-4d31-8e89-6971f7317144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:01.319378 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835919e-d5ff-4d31-8e89-6971f7317144 None admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:01.319698 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835919e-d5ff-4d31-8e89-6971f7317144 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:01.320019 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835919e-d5ff-4d31-8e89-6971f7317144 None admin] PATH_INFO: `/v3/projects/e646d320aa5a4f78bfccaf941cf46478/users/8c08fba2d7de42218400c5e652cc3de0/roles/6521132834a64b029695bbdd5eb251d2` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:01.338408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835919e-d5ff-4d31-8e89-6971f7317144 None admin] RBAC: Authorizing `identity:create_grant(project_id=e646d320aa5a4f78bfccaf941cf46478, user_id=8c08fba2d7de42218400c5e652cc3de0, role_id=6521132834a64b029695bbdd5eb251d2)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:01.339861 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835919e-d5ff-4d31-8e89-6971f7317144 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:01.353030 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 619/1232] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 27 04:12:01 2022] PUT /identity/v3/projects/e646d320aa5a4f78bfccaf941cf46478/users/8c08fba2d7de42218400c5e652cc3de0/roles/6521132834a64b029695bbdd5eb251d2 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:12:03.474905 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d52073-d97d-49a7-90a1-2273a7a63739 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:03.475382 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d52073-d97d-49a7-90a1-2273a7a63739 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:03.475499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d52073-d97d-49a7-90a1-2273a7a63739 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:03.476815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 614/1233] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:12:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:12:03.482526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c786f28d-175e-4f38-ac89-05e481acafb7 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:03.482975 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c786f28d-175e-4f38-ac89-05e481acafb7 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:03.483099 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c786f28d-175e-4f38-ac89-05e481acafb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:03.501967 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-c786f28d-175e-4f38-ac89-05e481acafb7 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:12:03.503298 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c786f28d-175e-4f38-ac89-05e481acafb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:12:03.573431 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 620/1234] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:12:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:12:03.599557 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d52073-d97d-49a7-90a1-2273a7a63739 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:03.628503 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:03.711678 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d52073-d97d-49a7-90a1-2273a7a63739 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:03.747815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad7be2-8e16-405e-9cfe-6a6031aa04d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:03.748548 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad7be2-8e16-405e-9cfe-6a6031aa04d0 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:03.748850 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad7be2-8e16-405e-9cfe-6a6031aa04d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:03.749107 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad7be2-8e16-405e-9cfe-6a6031aa04d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:03.760062 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ad7be2-8e16-405e-9cfe-6a6031aa04d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:12:03.766275 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-52ad7be2-8e16-405e-9cfe-6a6031aa04d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:12:03.766275 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:12:03.766275 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:12:03.766275 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:12:03.766275 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:12:03.766275 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:12:03.766275 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:12:03.766275 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:12:03.766275 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:12:03.766275 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:12:03.766275 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:12:03.766275 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:12:03.766275 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:12:03.766275 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:12:03.766275 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:12:03.766275 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:12:03.767439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:12:03.767439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:12:03.767439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:12:03.767439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:12:03.767439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:12:03.767439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:12:03.767439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:12:03.767439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:12:03.767439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:12:03.767439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:12:03.767439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:12:03.767439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:12:03.767439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:12:03.767439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:12:03.767439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:12:03.767439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:12:03.768506 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:12:03.768506 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:12:03.768506 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:12:03.768506 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:12:03.768506 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:12:03.768506 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:12:03.768506 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:12:03.768506 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:12:03.768506 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:12:03.768506 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:12:03.768506 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 04:12:03.768506 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:12:03.768506 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 615/1235] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 27 04:12:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:12:03.773548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c786f28d-175e-4f38-ac89-05e481acafb7 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:03.825205 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:03.886931 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c786f28d-175e-4f38-ac89-05e481acafb7 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:03.922494 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1bccea-5eaf-4b2f-8d58-12faa636bcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:03.923110 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1bccea-5eaf-4b2f-8d58-12faa636bcb9 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:03.923390 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1bccea-5eaf-4b2f-8d58-12faa636bcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:03.923659 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1bccea-5eaf-4b2f-8d58-12faa636bcb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:03.924373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1bccea-5eaf-4b2f-8d58-12faa636bcb9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:12:03.924673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1bccea-5eaf-4b2f-8d58-12faa636bcb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:03.926067 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1bccea-5eaf-4b2f-8d58-12faa636bcb9 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:03.935257 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 621/1236] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 27 04:12:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:12:03.941858 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad7be2-8e16-405e-9cfe-6a6031aa04d0 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:04.032439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:04.052131 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad7be2-8e16-405e-9cfe-6a6031aa04d0 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:04.093165 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f7b5c0-5e79-49fb-8e3e-96677b934052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:04.093814 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7b5c0-5e79-49fb-8e3e-96677b934052 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:04.094123 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7b5c0-5e79-49fb-8e3e-96677b934052 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:04.094396 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7b5c0-5e79-49fb-8e3e-96677b934052 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:04.105645 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f7b5c0-5e79-49fb-8e3e-96677b934052 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:04.108044 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f7b5c0-5e79-49fb-8e3e-96677b934052 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:04.124626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-b0f7b5c0-5e79-49fb-8e3e-96677b934052 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:12:04.124626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:12:04.124626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:12:04.124626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:12:04.124626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:12:04.124626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:12:04.124626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:12:04.124626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:12:04.124626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:12:04.124626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:12:04.124626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:12:04.124626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:12:04.124626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:12:04.124626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:12:04.124626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:12:04.125750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:12:04.125750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:12:04.125750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:12:04.125750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:12:04.125750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:12:04.125750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:12:04.125750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:12:04.125750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:12:04.125750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:12:04.125750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:12:04.125750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:12:04.125750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:12:04.125750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:12:04.125750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:12:04.125750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:12:04.125750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:12:04.125750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:12:04.126696 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:12:04.126696 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:12:04.126696 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:12:04.126696 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:12:04.126696 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 616/1237] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:12:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:12:04.131779 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1bccea-5eaf-4b2f-8d58-12faa636bcb9 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:04.253547 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1bccea-5eaf-4b2f-8d58-12faa636bcb9 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:04.264288 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:04.289814 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bee92b-359b-46a1-988e-8e39bd4cc96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:04.290394 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bee92b-359b-46a1-988e-8e39bd4cc96b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:04.290692 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bee92b-359b-46a1-988e-8e39bd4cc96b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:04.291020 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bee92b-359b-46a1-988e-8e39bd4cc96b None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:04.291664 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bee92b-359b-46a1-988e-8e39bd4cc96b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:12:04.291960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bee92b-359b-46a1-988e-8e39bd4cc96b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:04.293932 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bee92b-359b-46a1-988e-8e39bd4cc96b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:04.306038 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 622/1238] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:12:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:12:04.312598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f7b5c0-5e79-49fb-8e3e-96677b934052 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:04.422675 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f7b5c0-5e79-49fb-8e3e-96677b934052 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:04.460575 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e635c7-256c-4915-85c3-66339f20720d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:04.461238 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e635c7-256c-4915-85c3-66339f20720d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:04.461505 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e635c7-256c-4915-85c3-66339f20720d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:04.461784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e635c7-256c-4915-85c3-66339f20720d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:04.467519 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:04.473402 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e635c7-256c-4915-85c3-66339f20720d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:04.474911 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e635c7-256c-4915-85c3-66339f20720d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:04.486771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-07e635c7-256c-4915-85c3-66339f20720d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 04:12:04.486771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:12:04.486771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:12:04.486771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:12:04.486771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:12:04.486771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:12:04.486771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:12:04.486771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:12:04.486771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:12:04.486771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:12:04.486771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:12:04.486771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:12:04.486771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:12:04.486771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:12:04.486771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:12:04.486771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:12:04.488640 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:12:04.488640 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:12:04.488640 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:12:04.488640 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:12:04.488640 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:12:04.488640 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:12:04.488640 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:12:04.488640 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:12:04.488640 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:12:04.488640 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:12:04.488640 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:12:04.488640 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:12:04.488640 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:12:04.488640 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:12:04.488640 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:12:04.488640 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:12:04.488640 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:12:04.488640 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:12:04.490051 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:12:04.490051 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:12:04.490051 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:12:04.490051 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:12:04.490051 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:12:04.490051 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:12:04.490051 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:12:04.490051 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:12:04.490051 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 04:12:04.490051 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:12:04.490051 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 617/1239] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:12:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:12:04.493673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bee92b-359b-46a1-988e-8e39bd4cc96b None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:04.599886 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bee92b-359b-46a1-988e-8e39bd4cc96b None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:04.637308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeaefa2-3068-416d-bfdd-e719d4c2f261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:04.638135 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeaefa2-3068-416d-bfdd-e719d4c2f261 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:04.638457 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeaefa2-3068-416d-bfdd-e719d4c2f261 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:04.638690 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeaefa2-3068-416d-bfdd-e719d4c2f261 None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:04.639564 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeaefa2-3068-416d-bfdd-e719d4c2f261 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:12:04.639904 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeaefa2-3068-416d-bfdd-e719d4c2f261 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:04.641453 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeaefa2-3068-416d-bfdd-e719d4c2f261 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:04.661100 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 623/1240] 10.222.0.46 () {56 vars in 1143 bytes} [Wed Jul 27 04:12:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:12:04.667098 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e635c7-256c-4915-85c3-66339f20720d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:04.777172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e635c7-256c-4915-85c3-66339f20720d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:04.813501 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca12d7-a585-43d4-a712-7795905792bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:04.814419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca12d7-a585-43d4-a712-7795905792bf None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:04.814700 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca12d7-a585-43d4-a712-7795905792bf None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:04.814973 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca12d7-a585-43d4-a712-7795905792bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:04.882901 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:04.890964 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca12d7-a585-43d4-a712-7795905792bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:04.892573 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca12d7-a585-43d4-a712-7795905792bf None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:05.004200 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 618/1241] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:12:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 04:12:05.005201 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeaefa2-3068-416d-bfdd-e719d4c2f261 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:05.036649 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:05.119479 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeaefa2-3068-416d-bfdd-e719d4c2f261 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:05.157190 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be53278-f067-49d4-a787-491f8104a08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:05.157762 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be53278-f067-49d4-a787-491f8104a08f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:05.158076 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be53278-f067-49d4-a787-491f8104a08f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:05.158377 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be53278-f067-49d4-a787-491f8104a08f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:05.169162 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be53278-f067-49d4-a787-491f8104a08f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:05.170784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be53278-f067-49d4-a787-491f8104a08f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:05.186305 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-5be53278-f067-49d4-a787-491f8104a08f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:12:05.186305 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:12:05.186305 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:12:05.186305 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:12:05.186305 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:12:05.186305 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:12:05.186305 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:12:05.186305 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:12:05.186305 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:12:05.186305 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:12:05.186305 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:12:05.186305 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:12:05.186305 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:12:05.186305 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:12:05.186305 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:12:05.187442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:12:05.187442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:12:05.187442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:12:05.187442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:12:05.187442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:12:05.187442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:12:05.187442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:12:05.187442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:12:05.187442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:12:05.187442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:12:05.187442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:12:05.187442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:12:05.187442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:12:05.187442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:12:05.187442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:12:05.187442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:12:05.187442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:12:05.188341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:12:05.188341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:12:05.188341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:12:05.188341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:12:05.188341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 624/1242] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:12:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:12:05.194403 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca12d7-a585-43d4-a712-7795905792bf None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:05.309147 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:05.316401 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca12d7-a585-43d4-a712-7795905792bf None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:05.357377 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55fd784-6226-4dac-8383-05a7b471260d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:05.358036 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fd784-6226-4dac-8383-05a7b471260d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:05.358389 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fd784-6226-4dac-8383-05a7b471260d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:05.358723 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fd784-6226-4dac-8383-05a7b471260d None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:05.359711 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55fd784-6226-4dac-8383-05a7b471260d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:12:05.360056 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55fd784-6226-4dac-8383-05a7b471260d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:05.361659 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55fd784-6226-4dac-8383-05a7b471260d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:05.375769 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 619/1243] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:12:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:12:05.382518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be53278-f067-49d4-a787-491f8104a08f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:05.475531 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:05.498925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be53278-f067-49d4-a787-491f8104a08f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:05.535990 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6876e8-5026-4e74-9a2b-fcde0093cbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:05.536671 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6876e8-5026-4e74-9a2b-fcde0093cbb8 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:05.536996 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6876e8-5026-4e74-9a2b-fcde0093cbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:05.537273 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6876e8-5026-4e74-9a2b-fcde0093cbb8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:05.544664 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6876e8-5026-4e74-9a2b-fcde0093cbb8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:05.546134 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6876e8-5026-4e74-9a2b-fcde0093cbb8 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:05.559979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-9d6876e8-5026-4e74-9a2b-fcde0093cbb8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:12:05.559979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:12:05.559979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:12:05.559979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:12:05.559979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:12:05.559979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:12:05.559979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:12:05.559979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:12:05.559979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:12:05.559979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:12:05.559979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:12:05.559979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:12:05.559979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:12:05.559979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:12:05.559979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:12:05.561235 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:12:05.561235 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:12:05.561235 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:12:05.561235 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:12:05.561235 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:12:05.561235 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:12:05.561235 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:12:05.561235 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:12:05.561235 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:12:05.561235 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:12:05.561235 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:12:05.561235 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:12:05.561235 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:12:05.561235 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:12:05.561235 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:12:05.561235 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:12:05.561235 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:12:05.562187 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:12:05.562187 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:12:05.562187 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:12:05.562187 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:12:05.562187 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:12:05.562187 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:12:05.562187 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:12:05.562187 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:12:05.562187 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 625/1244] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:12:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:12:05.567711 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55fd784-6226-4dac-8383-05a7b471260d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:05.685076 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55fd784-6226-4dac-8383-05a7b471260d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:05.728081 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ade996-7f4f-4f5a-94f0-45f4ede4c554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:05.728735 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ade996-7f4f-4f5a-94f0-45f4ede4c554 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:05.729085 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ade996-7f4f-4f5a-94f0-45f4ede4c554 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:05.729506 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ade996-7f4f-4f5a-94f0-45f4ede4c554 None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:05.730570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ade996-7f4f-4f5a-94f0-45f4ede4c554 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:12:05.731452 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ade996-7f4f-4f5a-94f0-45f4ede4c554 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:05.733589 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ade996-7f4f-4f5a-94f0-45f4ede4c554 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:05.746489 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 620/1245] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:12:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:12:05.753912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6876e8-5026-4e74-9a2b-fcde0093cbb8 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:05.868029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6876e8-5026-4e74-9a2b-fcde0093cbb8 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:05.908752 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c0e65-ec6c-4ef4-9f79-711170ce093b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:05.909532 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c0e65-ec6c-4ef4-9f79-711170ce093b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:05.909837 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c0e65-ec6c-4ef4-9f79-711170ce093b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:05.910303 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c0e65-ec6c-4ef4-9f79-711170ce093b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:05.911162 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c0e65-ec6c-4ef4-9f79-711170ce093b None admin] RBAC: Adding query filter params (role.id=6521132834a64b029695bbdd5eb251d2, scope.project.id=e646d320aa5a4f78bfccaf941cf46478, user.id=8c08fba2d7de42218400c5e652cc3de0) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:12:05.911745 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c0e65-ec6c-4ef4-9f79-711170ce093b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:05.913999 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c0e65-ec6c-4ef4-9f79-711170ce093b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:05.922861 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 626/1246] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:12:05 2022] GET /identity/v3/role_assignments?user.id=8c08fba2d7de42218400c5e652cc3de0&scope.project.id=e646d320aa5a4f78bfccaf941cf46478&role.id=6521132834a64b029695bbdd5eb251d2 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:12:08.244633 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6a239-337e-421e-acca-ff8a6de93cd6 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:08.245521 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6a239-337e-421e-acca-ff8a6de93cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:08.245521 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6a239-337e-421e-acca-ff8a6de93cd6 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:08.247617 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 621/1247] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:12:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 04:12:08.253790 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3aada6-3832-401d-b4d1-f309add985fe None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:08.254510 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3aada6-3832-401d-b4d1-f309add985fe None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:08.254510 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3aada6-3832-401d-b4d1-f309add985fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:08.261263 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:08.274937 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-de3aada6-3832-401d-b4d1-f309add985fe None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:12:08.276158 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-de3aada6-3832-401d-b4d1-f309add985fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:12:08.353116 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 627/1248] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:12:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 04:12:08.377863 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6a239-337e-421e-acca-ff8a6de93cd6 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:08.400670 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:08.504052 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6a239-337e-421e-acca-ff8a6de93cd6 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:08.546867 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20740755-bfed-4527-8687-1ef632605777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:08.547695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20740755-bfed-4527-8687-1ef632605777 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:08.547982 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20740755-bfed-4527-8687-1ef632605777 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:08.548261 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20740755-bfed-4527-8687-1ef632605777 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:08.562367 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20740755-bfed-4527-8687-1ef632605777 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:12:08.567501 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-20740755-bfed-4527-8687-1ef632605777 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:12:08.567501 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:12:08.567501 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:12:08.567501 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:12:08.567501 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:12:08.567501 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:12:08.567501 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:12:08.567501 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:12:08.567501 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:12:08.567501 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:12:08.567501 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:12:08.567501 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:12:08.567501 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:12:08.567501 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:12:08.567501 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:12:08.567501 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:12:08.568646 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:12:08.568646 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:12:08.568646 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:12:08.568646 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:12:08.568646 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:12:08.568646 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:12:08.568646 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:12:08.568646 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:12:08.568646 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:12:08.568646 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:12:08.568646 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:12:08.568646 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:12:08.568646 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:12:08.568646 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:12:08.568646 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:12:08.568646 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:12:08.569606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:12:08.569606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:12:08.569606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:12:08.569606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:12:08.569606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:12:08.569606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:12:08.569606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:12:08.569606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:12:08.569606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:12:08.569606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:12:08.569606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:12:08.569606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:12:08.569606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 622/1249] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:12:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:12:08.576607 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3aada6-3832-401d-b4d1-f309add985fe None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:08.696046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3aada6-3832-401d-b4d1-f309add985fe None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:08.733360 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:08.738921 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020327e-dbe5-4c05-af4d-4f494401d97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:08.739919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020327e-dbe5-4c05-af4d-4f494401d97e None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:08.740280 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020327e-dbe5-4c05-af4d-4f494401d97e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:08.740632 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020327e-dbe5-4c05-af4d-4f494401d97e None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:08.741441 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1020327e-dbe5-4c05-af4d-4f494401d97e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:12:08.741754 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1020327e-dbe5-4c05-af4d-4f494401d97e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:08.743163 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1020327e-dbe5-4c05-af4d-4f494401d97e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:08.753733 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 628/1250] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:12:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:12:08.761026 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20740755-bfed-4527-8687-1ef632605777 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:08.851391 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:08.879763 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20740755-bfed-4527-8687-1ef632605777 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:08.920496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f889c1f-b4fb-4583-894b-61e02ca61659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:08.921311 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f889c1f-b4fb-4583-894b-61e02ca61659 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:08.921604 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f889c1f-b4fb-4583-894b-61e02ca61659 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:08.921885 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f889c1f-b4fb-4583-894b-61e02ca61659 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:08.936539 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f889c1f-b4fb-4583-894b-61e02ca61659 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:08.938597 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f889c1f-b4fb-4583-894b-61e02ca61659 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:08.957444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-9f889c1f-b4fb-4583-894b-61e02ca61659 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:12:08.957444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:12:08.957444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:12:08.957444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:12:08.957444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:12:08.957444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:12:08.957444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:12:08.957444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:12:08.957444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:12:08.957444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:12:08.957444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:12:08.957444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:12:08.957444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:12:08.957444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:12:08.957444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:12:08.958666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:12:08.958666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:12:08.958666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:12:08.958666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:12:08.958666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:12:08.958666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:12:08.958666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:12:08.958666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:12:08.958666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:12:08.958666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:12:08.958666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:12:08.958666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:12:08.958666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:12:08.958666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:12:08.958666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:12:08.958666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:12:08.958666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:12:08.960762 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:12:08.960762 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:12:08.960762 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:12:08.960762 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:12:08.960762 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 623/1251] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:12:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:12:08.966606 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020327e-dbe5-4c05-af4d-4f494401d97e None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:09.076584 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020327e-dbe5-4c05-af4d-4f494401d97e None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:09.120370 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2184fda9-771a-4926-8897-eff974d1ac4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:09.121033 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184fda9-771a-4926-8897-eff974d1ac4e None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:09.121350 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184fda9-771a-4926-8897-eff974d1ac4e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:09.121637 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184fda9-771a-4926-8897-eff974d1ac4e None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:09.122369 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2184fda9-771a-4926-8897-eff974d1ac4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:12:09.122679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2184fda9-771a-4926-8897-eff974d1ac4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:09.124173 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2184fda9-771a-4926-8897-eff974d1ac4e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:09.139299 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 629/1252] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:12:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:12:09.145493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f889c1f-b4fb-4583-894b-61e02ca61659 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:09.260531 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f889c1f-b4fb-4583-894b-61e02ca61659 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:09.291217 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:09.299284 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3207bcab-6552-4459-a830-48eeab573aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:09.299935 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3207bcab-6552-4459-a830-48eeab573aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:09.300217 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3207bcab-6552-4459-a830-48eeab573aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:09.300494 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3207bcab-6552-4459-a830-48eeab573aa2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:09.308137 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3207bcab-6552-4459-a830-48eeab573aa2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:09.309830 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3207bcab-6552-4459-a830-48eeab573aa2 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:09.321072 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-3207bcab-6552-4459-a830-48eeab573aa2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 04:12:09.321072 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:12:09.321072 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:12:09.321072 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:12:09.321072 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:12:09.321072 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:12:09.321072 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:12:09.321072 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:12:09.321072 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:12:09.321072 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:12:09.321072 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:12:09.321072 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:12:09.321072 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:12:09.321072 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:12:09.321072 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:12:09.321072 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:12:09.322101 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:12:09.322101 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:12:09.322101 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:12:09.322101 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:12:09.322101 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:12:09.322101 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:12:09.322101 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:12:09.322101 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:12:09.322101 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:12:09.322101 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:12:09.322101 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:12:09.322101 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:12:09.322101 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:12:09.322101 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:12:09.322101 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:12:09.322101 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:12:09.322101 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:12:09.322101 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:12:09.328902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:12:09.328902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:12:09.328902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:12:09.328902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:12:09.328902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:12:09.328902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:12:09.328902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:12:09.328902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:12:09.328902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 04:12:09.328902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:12:09.328902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 624/1253] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:12:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:12:09.329507 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2184fda9-771a-4926-8897-eff974d1ac4e None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:09.388288 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:09.441498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2184fda9-771a-4926-8897-eff974d1ac4e None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:09.483615 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9f79b-21f3-491f-afd7-62622a69e440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:09.484244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9f79b-21f3-491f-afd7-62622a69e440 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:09.484523 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9f79b-21f3-491f-afd7-62622a69e440 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:09.484801 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9f79b-21f3-491f-afd7-62622a69e440 None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:09.485604 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9f79b-21f3-491f-afd7-62622a69e440 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:12:09.485910 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9f79b-21f3-491f-afd7-62622a69e440 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:09.487442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9f79b-21f3-491f-afd7-62622a69e440 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:09.508485 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 630/1254] 10.222.0.46 () {56 vars in 1143 bytes} [Wed Jul 27 04:12:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:12:09.515161 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3207bcab-6552-4459-a830-48eeab573aa2 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:09.626653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3207bcab-6552-4459-a830-48eeab573aa2 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:09.665411 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e7829-6176-4d96-9b78-622dee7f9680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:09.666234 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e7829-6176-4d96-9b78-622dee7f9680 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:09.666524 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e7829-6176-4d96-9b78-622dee7f9680 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:09.666796 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e7829-6176-4d96-9b78-622dee7f9680 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:09.729634 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:09.743180 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e7829-6176-4d96-9b78-622dee7f9680 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:09.744656 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e7829-6176-4d96-9b78-622dee7f9680 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:09.845756 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 625/1255] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:12:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 27 04:12:09.851731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9f79b-21f3-491f-afd7-62622a69e440 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:09.962776 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9f79b-21f3-491f-afd7-62622a69e440 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:09.977010 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:10.002153 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e874addd-491b-401e-9d9c-0ce6db4123ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:10.002818 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874addd-491b-401e-9d9c-0ce6db4123ec None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:10.003082 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874addd-491b-401e-9d9c-0ce6db4123ec None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:10.003357 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874addd-491b-401e-9d9c-0ce6db4123ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:10.015548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e874addd-491b-401e-9d9c-0ce6db4123ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:10.017113 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e874addd-491b-401e-9d9c-0ce6db4123ec None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:10.033708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-e874addd-491b-401e-9d9c-0ce6db4123ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:12:10.033708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:12:10.033708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:12:10.033708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:12:10.033708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:12:10.033708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:12:10.033708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:12:10.033708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:12:10.033708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:12:10.033708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:12:10.033708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:12:10.033708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:12:10.033708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:12:10.033708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:12:10.033708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:12:10.034913 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:12:10.034913 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:12:10.034913 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:12:10.034913 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:12:10.034913 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:12:10.034913 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:12:10.034913 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:12:10.034913 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:12:10.034913 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:12:10.034913 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:12:10.034913 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:12:10.034913 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:12:10.034913 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:12:10.034913 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:12:10.034913 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:12:10.034913 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:12:10.034913 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:12:10.035937 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:12:10.035937 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:12:10.035937 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:12:10.035937 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:12:10.035937 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 631/1256] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:12:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:12:10.040792 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e7829-6176-4d96-9b78-622dee7f9680 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:10.134685 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:10.147941 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e7829-6176-4d96-9b78-622dee7f9680 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:10.182953 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc28cb-beaf-4624-a4a1-3c1d92ae7d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:10.183574 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc28cb-beaf-4624-a4a1-3c1d92ae7d97 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:10.184500 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc28cb-beaf-4624-a4a1-3c1d92ae7d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:10.184808 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc28cb-beaf-4624-a4a1-3c1d92ae7d97 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:10.185583 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc28cb-beaf-4624-a4a1-3c1d92ae7d97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:12:10.185888 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc28cb-beaf-4624-a4a1-3c1d92ae7d97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:10.187349 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc28cb-beaf-4624-a4a1-3c1d92ae7d97 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:10.199079 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 626/1257] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:12:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:12:10.205214 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e874addd-491b-401e-9d9c-0ce6db4123ec None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:10.325949 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e874addd-491b-401e-9d9c-0ce6db4123ec None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:10.362161 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e10e45-e30b-4f67-acf3-ff35e37557d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:10.362899 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e10e45-e30b-4f67-acf3-ff35e37557d0 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:10.363082 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e10e45-e30b-4f67-acf3-ff35e37557d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:10.363354 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e10e45-e30b-4f67-acf3-ff35e37557d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:10.370759 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e10e45-e30b-4f67-acf3-ff35e37557d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:10.372793 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e10e45-e30b-4f67-acf3-ff35e37557d0 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:10.384144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-23e10e45-e30b-4f67-acf3-ff35e37557d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:12:10.384144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:12:10.384144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:12:10.384144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:12:10.384144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:12:10.384144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:12:10.384144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:12:10.384144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:12:10.384144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:12:10.384144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:12:10.384144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:12:10.384144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:12:10.384144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:12:10.384144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:12:10.384144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:12:10.385263 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:12:10.385263 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:12:10.385263 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:12:10.385263 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:12:10.385263 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:12:10.385263 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:12:10.385263 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:12:10.385263 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:12:10.385263 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:12:10.385263 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:12:10.385263 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:12:10.385263 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:12:10.385263 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:12:10.385263 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:12:10.385263 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:12:10.385263 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:12:10.385263 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:12:10.386537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:12:10.386537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:12:10.386537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:12:10.386537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:12:10.386537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:12:10.386537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:12:10.386537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:12:10.386537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:12:10.386537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 632/1258] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:12:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:12:10.391420 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc28cb-beaf-4624-a4a1-3c1d92ae7d97 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:10.500545 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc28cb-beaf-4624-a4a1-3c1d92ae7d97 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:10.538087 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c485eaa-eba9-4e25-87b9-16c32d80096b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:10.538875 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c485eaa-eba9-4e25-87b9-16c32d80096b None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:10.539163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c485eaa-eba9-4e25-87b9-16c32d80096b None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:10.539445 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c485eaa-eba9-4e25-87b9-16c32d80096b None admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:10.542811 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c485eaa-eba9-4e25-87b9-16c32d80096b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:12:10.543874 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c485eaa-eba9-4e25-87b9-16c32d80096b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:10.545903 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c485eaa-eba9-4e25-87b9-16c32d80096b None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:10.561257 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 627/1259] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:12:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:12:10.567757 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e10e45-e30b-4f67-acf3-ff35e37557d0 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:10.595129 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:10.681820 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e10e45-e30b-4f67-acf3-ff35e37557d0 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:10.718642 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e2ee2-e70e-4b3d-b592-aa3f7cb5f87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:10.719283 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e2ee2-e70e-4b3d-b592-aa3f7cb5f87e None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:10.719563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e2ee2-e70e-4b3d-b592-aa3f7cb5f87e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:10.719836 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e2ee2-e70e-4b3d-b592-aa3f7cb5f87e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:10.720641 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8e2ee2-e70e-4b3d-b592-aa3f7cb5f87e None admin] RBAC: Adding query filter params (role.id=cf771a8e94ee46149eb081347fea6d67, scope.project.id=e646d320aa5a4f78bfccaf941cf46478, user.id=8c08fba2d7de42218400c5e652cc3de0) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:12:10.720948 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8e2ee2-e70e-4b3d-b592-aa3f7cb5f87e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:10.722429 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8e2ee2-e70e-4b3d-b592-aa3f7cb5f87e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:10.738022 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 633/1260] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:12:10 2022] GET /identity/v3/role_assignments?user.id=8c08fba2d7de42218400c5e652cc3de0&scope.project.id=e646d320aa5a4f78bfccaf941cf46478&role.id=cf771a8e94ee46149eb081347fea6d67 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:12:12.743105 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321496e-24f7-4039-b6c8-36e091d4339a None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:12.743604 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321496e-24f7-4039-b6c8-36e091d4339a None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:12.743604 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321496e-24f7-4039-b6c8-36e091d4339a None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:12.744934 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 628/1261] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:12:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:12:12.750917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252ada8-908e-4cec-bd54-fe0f5ab8f1cb None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:12.751337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252ada8-908e-4cec-bd54-fe0f5ab8f1cb None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:12.751480 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252ada8-908e-4cec-bd54-fe0f5ab8f1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:12.769931 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-e252ada8-908e-4cec-bd54-fe0f5ab8f1cb None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:12:12.771172 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e252ada8-908e-4cec-bd54-fe0f5ab8f1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:12:12.836107 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 634/1262] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:12:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:12:12.857825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321496e-24f7-4039-b6c8-36e091d4339a None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:12.868546 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:12.958722 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321496e-24f7-4039-b6c8-36e091d4339a None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:12.991962 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859efe6-e68e-455a-9551-05abbb523e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:12.992599 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859efe6-e68e-455a-9551-05abbb523e2f None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:12.992880 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859efe6-e68e-455a-9551-05abbb523e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:12.993202 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859efe6-e68e-455a-9551-05abbb523e2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:13.003887 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f859efe6-e68e-455a-9551-05abbb523e2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:12:13.008723 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-f859efe6-e68e-455a-9551-05abbb523e2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:12:13.008723 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:12:13.008723 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:12:13.008723 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:12:13.008723 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:12:13.008723 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:12:13.008723 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:12:13.008723 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:12:13.008723 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:12:13.008723 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:12:13.008723 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:12:13.008723 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:12:13.008723 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:12:13.008723 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:12:13.008723 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:12:13.008723 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:12:13.009796 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:12:13.009796 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:12:13.009796 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:12:13.009796 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:12:13.009796 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:12:13.009796 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:12:13.009796 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:12:13.009796 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:12:13.009796 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:12:13.009796 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:12:13.009796 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:12:13.009796 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:12:13.009796 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:12:13.009796 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:12:13.009796 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:12:13.009796 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:12:13.010977 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:12:13.010977 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:12:13.010977 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:12:13.010977 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:12:13.010977 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:12:13.010977 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:12:13.010977 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:12:13.010977 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:12:13.010977 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:12:13.010977 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:12:13.010977 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:12:13.010977 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:12:13.010977 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 629/1263] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:12:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:12:13.015975 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e252ada8-908e-4cec-bd54-fe0f5ab8f1cb None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:13.033715 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:13.116896 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e252ada8-908e-4cec-bd54-fe0f5ab8f1cb None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:13.149605 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9737f7bc-cc0e-407f-a975-847d58004b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:13.150248 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737f7bc-cc0e-407f-a975-847d58004b55 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:13.150533 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737f7bc-cc0e-407f-a975-847d58004b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:13.150811 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737f7bc-cc0e-407f-a975-847d58004b55 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:13.151528 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9737f7bc-cc0e-407f-a975-847d58004b55 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:12:13.151835 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9737f7bc-cc0e-407f-a975-847d58004b55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:13.153233 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9737f7bc-cc0e-407f-a975-847d58004b55 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:13.160994 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 635/1264] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:12:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:12:13.166773 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859efe6-e68e-455a-9551-05abbb523e2f None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:13.240170 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:13.280024 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859efe6-e68e-455a-9551-05abbb523e2f None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:13.312808 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff8256b-ea15-4e5c-89bc-f0b78a2ba786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:13.313443 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8256b-ea15-4e5c-89bc-f0b78a2ba786 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:13.313719 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8256b-ea15-4e5c-89bc-f0b78a2ba786 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:13.314279 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8256b-ea15-4e5c-89bc-f0b78a2ba786 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:13.324307 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff8256b-ea15-4e5c-89bc-f0b78a2ba786 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:13.325757 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff8256b-ea15-4e5c-89bc-f0b78a2ba786 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:13.339340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-8ff8256b-ea15-4e5c-89bc-f0b78a2ba786 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:12:13.339340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:12:13.339340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:12:13.339340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:12:13.339340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:12:13.339340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:12:13.339340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:12:13.339340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:12:13.339340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:12:13.339340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:12:13.339340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:12:13.339340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:12:13.339340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:12:13.339340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:12:13.339340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:12:13.340771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:12:13.340771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:12:13.340771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:12:13.340771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:12:13.340771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:12:13.340771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:12:13.340771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:12:13.340771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:12:13.340771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:12:13.340771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:12:13.340771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:12:13.340771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:12:13.340771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:12:13.340771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:12:13.340771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:12:13.340771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:12:13.340771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:12:13.341876 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:12:13.341876 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:12:13.341876 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:12:13.341876 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:12:13.341876 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 630/1265] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:12:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:12:13.346446 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9737f7bc-cc0e-407f-a975-847d58004b55 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:13.424926 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:13.444651 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9737f7bc-cc0e-407f-a975-847d58004b55 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:13.477776 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4bf12-500f-4e06-89f3-5059919c6fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:13.478434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4bf12-500f-4e06-89f3-5059919c6fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:13.478718 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4bf12-500f-4e06-89f3-5059919c6fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:13.478993 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4bf12-500f-4e06-89f3-5059919c6fc4 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:13.479684 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e4bf12-500f-4e06-89f3-5059919c6fc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:12:13.479988 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e4bf12-500f-4e06-89f3-5059919c6fc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:13.482005 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e4bf12-500f-4e06-89f3-5059919c6fc4 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:13.493267 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 636/1266] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:12:13.499137 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff8256b-ea15-4e5c-89bc-f0b78a2ba786 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:13.599304 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff8256b-ea15-4e5c-89bc-f0b78a2ba786 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:13.618558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:13.633237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4bf0b4-bfd4-4cd3-9d0f-8311807a067d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:13.633869 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bf0b4-bfd4-4cd3-9d0f-8311807a067d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:13.634189 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bf0b4-bfd4-4cd3-9d0f-8311807a067d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:13.634468 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bf0b4-bfd4-4cd3-9d0f-8311807a067d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:13.642727 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4bf0b4-bfd4-4cd3-9d0f-8311807a067d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:13.644158 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4bf0b4-bfd4-4cd3-9d0f-8311807a067d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:13.655535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-ba4bf0b4-bfd4-4cd3-9d0f-8311807a067d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 04:12:13.655535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:12:13.655535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:12:13.655535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:12:13.655535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:12:13.655535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:12:13.655535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:12:13.655535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:12:13.655535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:12:13.655535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:12:13.655535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:12:13.655535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:12:13.655535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:12:13.655535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:12:13.655535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:12:13.655535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:12:13.656608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:12:13.656608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:12:13.656608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:12:13.656608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:12:13.656608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:12:13.656608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:12:13.656608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:12:13.656608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:12:13.656608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:12:13.656608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:12:13.656608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:12:13.656608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:12:13.656608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:12:13.656608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:12:13.656608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:12:13.656608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:12:13.656608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:12:13.656608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:12:13.657925 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:12:13.657925 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:12:13.657925 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:12:13.657925 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:12:13.657925 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:12:13.657925 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:12:13.657925 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:12:13.657925 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:12:13.657925 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 04:12:13.657925 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:12:13.657925 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 631/1267] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:12:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:12:13.662588 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4bf12-500f-4e06-89f3-5059919c6fc4 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:13.759920 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4bf12-500f-4e06-89f3-5059919c6fc4 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:13.792478 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800452ba-5b1c-4605-aee6-ff7a18d09af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:13.793639 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800452ba-5b1c-4605-aee6-ff7a18d09af8 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:13.794187 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800452ba-5b1c-4605-aee6-ff7a18d09af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:13.794525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800452ba-5b1c-4605-aee6-ff7a18d09af8 None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:13.795324 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800452ba-5b1c-4605-aee6-ff7a18d09af8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:12:13.795635 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800452ba-5b1c-4605-aee6-ff7a18d09af8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:13.797071 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800452ba-5b1c-4605-aee6-ff7a18d09af8 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:13.805502 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:13.815100 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 637/1268] 10.222.0.46 () {56 vars in 1143 bytes} [Wed Jul 27 04:12:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:12:13.820848 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4bf0b4-bfd4-4cd3-9d0f-8311807a067d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:13.917061 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4bf0b4-bfd4-4cd3-9d0f-8311807a067d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:13.950470 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e003c-ab2c-400d-9fea-766a784f42dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:13.951056 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e003c-ab2c-400d-9fea-766a784f42dd None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:13.951339 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e003c-ab2c-400d-9fea-766a784f42dd None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:13.951611 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e003c-ab2c-400d-9fea-766a784f42dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:13.961781 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e003c-ab2c-400d-9fea-766a784f42dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:13.963256 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e003c-ab2c-400d-9fea-766a784f42dd None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:13.977077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-645e003c-ab2c-400d-9fea-766a784f42dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:12:13.977077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:12:13.977077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:12:13.977077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:12:13.977077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:12:13.977077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:12:13.977077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:12:13.977077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:12:13.977077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:12:13.977077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:12:13.977077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:12:13.977077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:12:13.977077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:12:13.977077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:12:13.977077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:12:13.978173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:12:13.978173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:12:13.978173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:12:13.978173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:12:13.978173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:12:13.978173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:12:13.978173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:12:13.978173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:12:13.978173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:12:13.978173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:12:13.978173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:12:13.978173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:12:13.978173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:12:13.978173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:12:13.978173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:12:13.978173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:12:13.978173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:12:13.979096 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:12:13.979096 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:12:13.979096 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:12:13.979096 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:12:13.979096 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 632/1269] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:12:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:12:13.984434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800452ba-5b1c-4605-aee6-ff7a18d09af8 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:14.080106 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800452ba-5b1c-4605-aee6-ff7a18d09af8 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:14.113874 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7e397-824a-4647-8cf5-8291b4c9d556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:14.114518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7e397-824a-4647-8cf5-8291b4c9d556 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:14.114800 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7e397-824a-4647-8cf5-8291b4c9d556 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:14.115077 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7e397-824a-4647-8cf5-8291b4c9d556 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:14.115771 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c7e397-824a-4647-8cf5-8291b4c9d556 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:12:14.116075 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c7e397-824a-4647-8cf5-8291b4c9d556 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:14.117428 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c7e397-824a-4647-8cf5-8291b4c9d556 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:14.128947 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 638/1270] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:12:14.134398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e003c-ab2c-400d-9fea-766a784f42dd None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:14.165280 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:14.238397 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e003c-ab2c-400d-9fea-766a784f42dd None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:14.270951 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00d38c-a749-4990-86ad-54bced62eb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:14.271591 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00d38c-a749-4990-86ad-54bced62eb69 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:14.271877 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00d38c-a749-4990-86ad-54bced62eb69 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:14.272161 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00d38c-a749-4990-86ad-54bced62eb69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:14.279249 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae00d38c-a749-4990-86ad-54bced62eb69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:14.280700 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae00d38c-a749-4990-86ad-54bced62eb69 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:14.291773 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-ae00d38c-a749-4990-86ad-54bced62eb69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:12:14.291773 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:12:14.291773 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:12:14.291773 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:12:14.291773 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:12:14.291773 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:12:14.291773 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:12:14.291773 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:12:14.291773 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:12:14.291773 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:12:14.291773 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:12:14.291773 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:12:14.291773 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:12:14.291773 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:12:14.291773 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:12:14.292811 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:12:14.292811 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:12:14.292811 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:12:14.292811 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:12:14.292811 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:12:14.292811 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:12:14.292811 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:12:14.292811 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:12:14.292811 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:12:14.292811 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:12:14.292811 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:12:14.292811 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:12:14.292811 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:12:14.292811 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:12:14.292811 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:12:14.292811 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:12:14.292811 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:12:14.294907 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:12:14.294907 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:12:14.294907 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:12:14.294907 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:12:14.294907 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:12:14.294907 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:12:14.294907 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:12:14.294907 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:12:14.294907 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 633/1271] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:12:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:12:14.299319 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7e397-824a-4647-8cf5-8291b4c9d556 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:14.356404 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:14.396870 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7e397-824a-4647-8cf5-8291b4c9d556 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:14.430551 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd91e0-3e47-4e86-af0c-a170a2c86d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:14.431166 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd91e0-3e47-4e86-af0c-a170a2c86d90 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:14.431448 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd91e0-3e47-4e86-af0c-a170a2c86d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:14.431778 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd91e0-3e47-4e86-af0c-a170a2c86d90 None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:14.432540 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd91e0-3e47-4e86-af0c-a170a2c86d90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:12:14.432850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd91e0-3e47-4e86-af0c-a170a2c86d90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:14.434460 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd91e0-3e47-4e86-af0c-a170a2c86d90 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:14.446346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 639/1272] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:12:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:12:14.452618 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00d38c-a749-4990-86ad-54bced62eb69 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:14.543180 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:14.552110 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00d38c-a749-4990-86ad-54bced62eb69 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:14.585270 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78258940-2836-4383-96ff-4cf8bdbcd5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:14.585933 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78258940-2836-4383-96ff-4cf8bdbcd5df None admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:14.586250 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78258940-2836-4383-96ff-4cf8bdbcd5df None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:14.586530 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78258940-2836-4383-96ff-4cf8bdbcd5df None admin] PATH_INFO: `/v3/projects/e646d320aa5a4f78bfccaf941cf46478/users/8c08fba2d7de42218400c5e652cc3de0/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:14.588441 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78258940-2836-4383-96ff-4cf8bdbcd5df None admin] RBAC: Authorizing `identity:create_grant(project_id=e646d320aa5a4f78bfccaf941cf46478, user_id=8c08fba2d7de42218400c5e652cc3de0, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:14.590529 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78258940-2836-4383-96ff-4cf8bdbcd5df None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:14.602683 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 634/1273] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 27 04:12:14 2022] PUT /identity/v3/projects/e646d320aa5a4f78bfccaf941cf46478/users/8c08fba2d7de42218400c5e652cc3de0/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:12:16.555847 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b41cd-ce37-4eca-862f-a2a7e5c86aad None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:16.556394 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b41cd-ce37-4eca-862f-a2a7e5c86aad None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:16.556394 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b41cd-ce37-4eca-862f-a2a7e5c86aad None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:16.557752 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 640/1274] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:12:16.563320 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257bf908-dfba-4655-9bcb-415c6775a2dd None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:16.563717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257bf908-dfba-4655-9bcb-415c6775a2dd None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:16.564127 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257bf908-dfba-4655-9bcb-415c6775a2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:16.582326 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-257bf908-dfba-4655-9bcb-415c6775a2dd None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:12:16.583496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-257bf908-dfba-4655-9bcb-415c6775a2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:12:16.647086 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 635/1275] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:12:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:12:16.669839 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b41cd-ce37-4eca-862f-a2a7e5c86aad None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:16.767492 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b41cd-ce37-4eca-862f-a2a7e5c86aad None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:16.783739 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:16.800544 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a587ed9-acbe-4af8-b4ca-3cb14ea9293e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:16.801175 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a587ed9-acbe-4af8-b4ca-3cb14ea9293e None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:16.801457 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a587ed9-acbe-4af8-b4ca-3cb14ea9293e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:16.801732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a587ed9-acbe-4af8-b4ca-3cb14ea9293e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:16.812353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a587ed9-acbe-4af8-b4ca-3cb14ea9293e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 04:12:16.817546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-5a587ed9-acbe-4af8-b4ca-3cb14ea9293e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:12:16.817546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:12:16.817546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:12:16.817546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:12:16.817546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:12:16.817546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:12:16.817546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:12:16.817546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:12:16.817546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:12:16.817546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:12:16.817546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:12:16.817546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:12:16.817546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 04:12:16.817546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 04:12:16.817546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 04:12:16.817546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 04:12:16.818627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 04:12:16.818627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 04:12:16.818627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 04:12:16.818627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 04:12:16.818627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:12:16.818627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:12:16.818627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:12:16.818627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:12:16.818627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:12:16.818627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:12:16.818627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:12:16.818627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:12:16.818627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:12:16.818627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:12:16.818627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:12:16.818627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:12:16.819525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:12:16.819525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:12:16.819525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:12:16.819525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:12:16.819525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 04:12:16.819525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 04:12:16.819525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 04:12:16.819525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 04:12:16.819525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 04:12:16.819525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 04:12:16.819525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 04:12:16.819525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:12:16.819525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 641/1276] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 27 04:12:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:12:16.824783 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257bf908-dfba-4655-9bcb-415c6775a2dd None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:16.902708 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:16.922354 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257bf908-dfba-4655-9bcb-415c6775a2dd None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:16.955269 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c852c42-718a-43a6-b66e-d3f21b12c5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:16.955875 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c852c42-718a-43a6-b66e-d3f21b12c5c6 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:16.956179 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c852c42-718a-43a6-b66e-d3f21b12c5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:16.956452 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c852c42-718a-43a6-b66e-d3f21b12c5c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:16.957161 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c852c42-718a-43a6-b66e-d3f21b12c5c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:12:16.957460 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c852c42-718a-43a6-b66e-d3f21b12c5c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:16.958837 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c852c42-718a-43a6-b66e-d3f21b12c5c6 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:16.966541 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 636/1277] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 27 04:12:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:12:16.972185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a587ed9-acbe-4af8-b4ca-3cb14ea9293e None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:17.070582 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a587ed9-acbe-4af8-b4ca-3cb14ea9293e None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:17.105168 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4754b51-cec0-4f1c-a914-bc0541bacc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:17.106352 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4754b51-cec0-4f1c-a914-bc0541bacc69 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:17.106637 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4754b51-cec0-4f1c-a914-bc0541bacc69 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:17.106913 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4754b51-cec0-4f1c-a914-bc0541bacc69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:17.116711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4754b51-cec0-4f1c-a914-bc0541bacc69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:17.118176 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4754b51-cec0-4f1c-a914-bc0541bacc69 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:17.132866 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-c4754b51-cec0-4f1c-a914-bc0541bacc69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:12:17.132866 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:12:17.132866 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:12:17.132866 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:12:17.132866 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:12:17.132866 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:12:17.132866 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:12:17.132866 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:12:17.132866 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:12:17.132866 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:12:17.132866 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:12:17.132866 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:12:17.132866 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:12:17.132866 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:12:17.132866 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:12:17.133894 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:12:17.133894 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:12:17.133894 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:12:17.133894 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:12:17.133894 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:12:17.133894 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:12:17.133894 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:12:17.133894 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:12:17.133894 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:12:17.133894 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:12:17.133894 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:12:17.133894 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:12:17.133894 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:12:17.133894 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:12:17.133894 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:12:17.133894 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:12:17.133894 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:12:17.135168 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:12:17.135168 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:12:17.135168 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:12:17.135168 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:12:17.135168 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 642/1278] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:12:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:12:17.139832 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c852c42-718a-43a6-b66e-d3f21b12c5c6 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:17.247319 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c852c42-718a-43a6-b66e-d3f21b12c5c6 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:17.279564 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669a54a-baf8-4e7b-9055-d1595b2bef70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:17.280179 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669a54a-baf8-4e7b-9055-d1595b2bef70 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:17.280457 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669a54a-baf8-4e7b-9055-d1595b2bef70 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:17.280727 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669a54a-baf8-4e7b-9055-d1595b2bef70 None admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:17.281418 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7669a54a-baf8-4e7b-9055-d1595b2bef70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:12:17.281718 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7669a54a-baf8-4e7b-9055-d1595b2bef70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:17.283201 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7669a54a-baf8-4e7b-9055-d1595b2bef70 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:17.295045 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 637/1279] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:12:17.300695 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4754b51-cec0-4f1c-a914-bc0541bacc69 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:17.330501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:17.396979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4754b51-cec0-4f1c-a914-bc0541bacc69 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:17.429471 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fa00e-11bf-4860-9c72-dcfb0ba12e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:17.430213 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fa00e-11bf-4860-9c72-dcfb0ba12e44 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:17.430491 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fa00e-11bf-4860-9c72-dcfb0ba12e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:17.430762 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fa00e-11bf-4860-9c72-dcfb0ba12e44 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:17.439139 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23fa00e-11bf-4860-9c72-dcfb0ba12e44 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:17.440559 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23fa00e-11bf-4860-9c72-dcfb0ba12e44 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:17.451548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-e23fa00e-11bf-4860-9c72-dcfb0ba12e44 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 04:12:17.451548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:12:17.451548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:12:17.451548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:12:17.451548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:12:17.451548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:12:17.451548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:12:17.451548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:12:17.451548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:12:17.451548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:12:17.451548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:12:17.451548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:12:17.451548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 04:12:17.451548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 04:12:17.451548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 04:12:17.451548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 04:12:17.452656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:12:17.452656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:12:17.452656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 04:12:17.452656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:12:17.452656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 04:12:17.452656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 04:12:17.452656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:12:17.452656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:12:17.452656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:12:17.452656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:12:17.452656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:12:17.452656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:12:17.452656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:12:17.452656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:12:17.452656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:12:17.452656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:12:17.452656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:12:17.452656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:12:17.455237 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:12:17.455237 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:12:17.455237 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 04:12:17.455237 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 04:12:17.455237 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 04:12:17.455237 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 04:12:17.455237 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 04:12:17.455237 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 04:12:17.455237 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 04:12:17.455237 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:12:17.455237 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 643/1280] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:12:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:12:17.458444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669a54a-baf8-4e7b-9055-d1595b2bef70 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:17.464082 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:17.555418 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669a54a-baf8-4e7b-9055-d1595b2bef70 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:17.588235 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86060d-a95c-4c92-85f7-4e7d3d6e4833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:17.588844 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86060d-a95c-4c92-85f7-4e7d3d6e4833 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:17.589125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86060d-a95c-4c92-85f7-4e7d3d6e4833 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:17.589397 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86060d-a95c-4c92-85f7-4e7d3d6e4833 None admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:17.590850 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b86060d-a95c-4c92-85f7-4e7d3d6e4833 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:12:17.591210 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b86060d-a95c-4c92-85f7-4e7d3d6e4833 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:17.592650 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b86060d-a95c-4c92-85f7-4e7d3d6e4833 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:17.609671 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 638/1281] 10.222.0.46 () {56 vars in 1143 bytes} [Wed Jul 27 04:12:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:12:17.615482 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fa00e-11bf-4860-9c72-dcfb0ba12e44 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:17.702543 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:17.712372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fa00e-11bf-4860-9c72-dcfb0ba12e44 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:17.744958 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1245e-a8ec-4c61-a62a-7b41a869eb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:17.745568 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1245e-a8ec-4c61-a62a-7b41a869eb0d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:17.745850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1245e-a8ec-4c61-a62a-7b41a869eb0d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:17.746795 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1245e-a8ec-4c61-a62a-7b41a869eb0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:17.813461 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf1245e-a8ec-4c61-a62a-7b41a869eb0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:17.815190 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf1245e-a8ec-4c61-a62a-7b41a869eb0d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:17.899717 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:17.910767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 644/1282] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 27 04:12:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 04:12:17.916714 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86060d-a95c-4c92-85f7-4e7d3d6e4833 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:17.977962 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:18.021041 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86060d-a95c-4c92-85f7-4e7d3d6e4833 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:18.056396 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69779a1a-67ad-4834-8573-193ee4969a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:18.057035 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69779a1a-67ad-4834-8573-193ee4969a6b None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:18.057318 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69779a1a-67ad-4834-8573-193ee4969a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:18.057594 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69779a1a-67ad-4834-8573-193ee4969a6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:18.067551 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69779a1a-67ad-4834-8573-193ee4969a6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:18.068998 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69779a1a-67ad-4834-8573-193ee4969a6b None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:18.083171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-69779a1a-67ad-4834-8573-193ee4969a6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:12:18.083171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:12:18.083171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:12:18.083171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:12:18.083171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:12:18.083171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:12:18.083171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:12:18.083171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:12:18.083171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:12:18.083171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:12:18.083171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:12:18.083171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:12:18.083171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 04:12:18.083171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 04:12:18.083171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 04:12:18.084251 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 04:12:18.084251 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:12:18.084251 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:12:18.084251 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:12:18.084251 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:12:18.084251 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:12:18.084251 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:12:18.084251 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:12:18.084251 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:12:18.084251 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:12:18.084251 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:12:18.084251 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:12:18.084251 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:12:18.084251 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:12:18.084251 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:12:18.084251 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:12:18.084251 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:12:18.085367 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 04:12:18.085367 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 04:12:18.085367 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 04:12:18.085367 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:12:18.085367 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 639/1283] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:12:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:12:18.090664 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1245e-a8ec-4c61-a62a-7b41a869eb0d None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:18.187029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1245e-a8ec-4c61-a62a-7b41a869eb0d None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:18.226270 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c38d1-7ac2-468c-a325-ae6782a27a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:18.226890 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c38d1-7ac2-468c-a325-ae6782a27a45 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:18.227174 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c38d1-7ac2-468c-a325-ae6782a27a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:18.227452 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c38d1-7ac2-468c-a325-ae6782a27a45 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:18.228152 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c38d1-7ac2-468c-a325-ae6782a27a45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:12:18.228462 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c38d1-7ac2-468c-a325-ae6782a27a45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:18.230116 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c38d1-7ac2-468c-a325-ae6782a27a45 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:18.241746 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 645/1284] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 27 04:12:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:12:18.247418 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69779a1a-67ad-4834-8573-193ee4969a6b None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:18.345676 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69779a1a-67ad-4834-8573-193ee4969a6b None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:18.358764 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:18.378220 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda38293-d8c1-4622-928f-c0ae59a51d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:18.378853 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda38293-d8c1-4622-928f-c0ae59a51d5d None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:18.379134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda38293-d8c1-4622-928f-c0ae59a51d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:18.379410 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda38293-d8c1-4622-928f-c0ae59a51d5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:18.385796 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda38293-d8c1-4622-928f-c0ae59a51d5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:18.387917 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda38293-d8c1-4622-928f-c0ae59a51d5d None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:18.398868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application [None req-bda38293-d8c1-4622-928f-c0ae59a51d5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:12:18.398868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:12:18.398868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:12:18.398868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:12:18.398868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:12:18.398868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:12:18.398868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:12:18.398868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:12:18.398868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:12:18.398868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:12:18.398868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:12:18.398868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:12:18.398868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 04:12:18.398868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 04:12:18.398868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 04:12:18.399901 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 04:12:18.399901 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:12:18.399901 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:12:18.399901 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:12:18.399901 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:12:18.399901 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:12:18.399901 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:12:18.399901 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:12:18.399901 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application with Lock( Jul 27 04:12:18.399901 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:12:18.399901 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:12:18.399901 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:12:18.399901 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:12:18.399901 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:12:18.399901 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:12:18.399901 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:12:18.399901 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:12:18.401015 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 04:12:18.401015 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 04:12:18.401015 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 04:12:18.401015 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 04:12:18.401015 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 04:12:18.401015 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 04:12:18.401015 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 04:12:18.401015 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: ERROR keystone.server.flask.application Jul 27 04:12:18.401015 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 640/1285] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 27 04:12:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:12:18.406398 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c38d1-7ac2-468c-a325-ae6782a27a45 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:18.454864 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:18.503864 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c38d1-7ac2-468c-a325-ae6782a27a45 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:18.536552 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf6cc4b-641e-47f9-85a1-5aac169d5933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:18.537176 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6cc4b-641e-47f9-85a1-5aac169d5933 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:18.537458 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6cc4b-641e-47f9-85a1-5aac169d5933 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:18.537731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6cc4b-641e-47f9-85a1-5aac169d5933 None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:18.538513 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf6cc4b-641e-47f9-85a1-5aac169d5933 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:12:18.538824 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf6cc4b-641e-47f9-85a1-5aac169d5933 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:18.540236 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf6cc4b-641e-47f9-85a1-5aac169d5933 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:18.551730 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 646/1286] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 27 04:12:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:12:18.557927 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda38293-d8c1-4622-928f-c0ae59a51d5d None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:18.657279 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda38293-d8c1-4622-928f-c0ae59a51d5d None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:18.690154 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2adb8c-652b-4eed-9987-f32711d38bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:18.690756 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2adb8c-652b-4eed-9987-f32711d38bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:18.691038 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2adb8c-652b-4eed-9987-f32711d38bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:18.691313 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2adb8c-652b-4eed-9987-f32711d38bc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:18.692099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2adb8c-652b-4eed-9987-f32711d38bc9 None admin] RBAC: Adding query filter params (role.id=cf771a8e94ee46149eb081347fea6d67, scope.project.id=e646d320aa5a4f78bfccaf941cf46478, user.id=8c08fba2d7de42218400c5e652cc3de0) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:12:18.692411 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2adb8c-652b-4eed-9987-f32711d38bc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:18.693838 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2adb8c-652b-4eed-9987-f32711d38bc9 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:18.700273 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 641/1287] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 27 04:12:18 2022] GET /identity/v3/role_assignments?user.id=8c08fba2d7de42218400c5e652cc3de0&scope.project.id=e646d320aa5a4f78bfccaf941cf46478&role.id=cf771a8e94ee46149eb081347fea6d67 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:12:20.711955 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348841e-64e8-46b8-a6f2-53e9c3f316d1 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:20.712450 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348841e-64e8-46b8-a6f2-53e9c3f316d1 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:20.712450 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348841e-64e8-46b8-a6f2-53e9c3f316d1 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:20.713780 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 647/1288] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:12:20.718962 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa4358-b2c7-4a45-9ad8-5a5b0dcfeaba None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:20.719349 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa4358-b2c7-4a45-9ad8-5a5b0dcfeaba None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:20.719554 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa4358-b2c7-4a45-9ad8-5a5b0dcfeaba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:20.738067 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-53aa4358-b2c7-4a45-9ad8-5a5b0dcfeaba None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:12:20.739247 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-53aa4358-b2c7-4a45-9ad8-5a5b0dcfeaba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:12:20.768559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:20.805498 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 642/1289] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:12:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:12:20.828421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a348841e-64e8-46b8-a6f2-53e9c3f316d1 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:20.928173 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a348841e-64e8-46b8-a6f2-53e9c3f316d1 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:20.939525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:20.962684 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d422f-882c-47f6-8863-4dd1d1ea1d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:20.963270 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d422f-882c-47f6-8863-4dd1d1ea1d89 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:20.963553 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d422f-882c-47f6-8863-4dd1d1ea1d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:20.963834 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d422f-882c-47f6-8863-4dd1d1ea1d89 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:20.964468 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d422f-882c-47f6-8863-4dd1d1ea1d89 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:20.966571 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d422f-882c-47f6-8863-4dd1d1ea1d89 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:20.976410 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-785d422f-882c-47f6-8863-4dd1d1ea1d89 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 04:12:20.976410 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:12:20.976410 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 04:12:20.976410 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 04:12:20.976410 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 04:12:20.976410 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 04:12:20.976410 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 04:12:20.976410 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:12:20.976410 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 04:12:20.976410 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:12:20.976410 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 04:12:20.976410 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:12:20.976410 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:12:20.976410 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:12:20.976410 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:12:20.976410 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:12:20.976410 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:12:20.977623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:12:20.977623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:12:20.977623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:12:20.977623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:12:20.977623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:12:20.977623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:12:20.977623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 04:12:20.977623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 04:12:20.977623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 04:12:20.977623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 04:12:20.977623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:12:20.977623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:12:20.977623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:12:20.977623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:12:20.977623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:12:20.977623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:12:20.978623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:12:20.978623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:12:20.978623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:12:20.978623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:12:20.978623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:12:20.978623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:12:20.978623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:12:20.978623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:12:20.978623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:12:20.978623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:12:20.978623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 04:12:20.978623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:12:20.978623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 04:12:20.978623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:12:20.978623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 648/1290] 10.222.0.46 () {56 vars in 1117 bytes} [Wed Jul 27 04:12:20 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:12:20.984486 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa4358-b2c7-4a45-9ad8-5a5b0dcfeaba None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:21.079696 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa4358-b2c7-4a45-9ad8-5a5b0dcfeaba None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:21.116642 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df6764d-6116-4321-a6f0-cf92d025536b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:21.117257 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df6764d-6116-4321-a6f0-cf92d025536b None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:21.117534 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df6764d-6116-4321-a6f0-cf92d025536b None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:21.117806 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df6764d-6116-4321-a6f0-cf92d025536b None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:21.118518 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df6764d-6116-4321-a6f0-cf92d025536b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:12:21.118818 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df6764d-6116-4321-a6f0-cf92d025536b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:21.120179 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df6764d-6116-4321-a6f0-cf92d025536b None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:21.124936 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 643/1291] 10.222.0.46 () {56 vars in 1116 bytes} [Wed Jul 27 04:12:20 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:12:21.130674 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d422f-882c-47f6-8863-4dd1d1ea1d89 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:21.234117 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d422f-882c-47f6-8863-4dd1d1ea1d89 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:21.272659 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e9a202-5689-4dc8-9a16-17a06e4e9298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:21.273347 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9a202-5689-4dc8-9a16-17a06e4e9298 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:21.273677 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9a202-5689-4dc8-9a16-17a06e4e9298 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:21.274072 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9a202-5689-4dc8-9a16-17a06e4e9298 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:21.274834 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e9a202-5689-4dc8-9a16-17a06e4e9298 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:12:21.275229 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e9a202-5689-4dc8-9a16-17a06e4e9298 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:21.276922 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e9a202-5689-4dc8-9a16-17a06e4e9298 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:21.282939 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 649/1292] 10.222.0.46 () {56 vars in 1116 bytes} [Wed Jul 27 04:12:21 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:12:23.276477 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5aded-7aa2-4698-bd90-e697f8ed9724 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:23.277065 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5aded-7aa2-4698-bd90-e697f8ed9724 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:23.277065 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5aded-7aa2-4698-bd90-e697f8ed9724 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:23.278742 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 644/1293] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:12:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:12:23.283721 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3b72d-f74b-48d3-8735-8d6b078cffff None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:23.284162 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3b72d-f74b-48d3-8735-8d6b078cffff None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:23.284256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3b72d-f74b-48d3-8735-8d6b078cffff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:23.301737 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-ada3b72d-f74b-48d3-8735-8d6b078cffff None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:12:23.302954 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-ada3b72d-f74b-48d3-8735-8d6b078cffff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:12:23.332552 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:23.368711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 650/1294] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:12:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:12:23.391890 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c5aded-7aa2-4698-bd90-e697f8ed9724 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:23.422403 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:23.494752 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c5aded-7aa2-4698-bd90-e697f8ed9724 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:23.528585 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d5a807-114b-4a8a-8228-f5e47a48f0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:23.529133 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d5a807-114b-4a8a-8228-f5e47a48f0c1 None admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:23.529414 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d5a807-114b-4a8a-8228-f5e47a48f0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:23.529686 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d5a807-114b-4a8a-8228-f5e47a48f0c1 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:23.530421 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d5a807-114b-4a8a-8228-f5e47a48f0c1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:23.531813 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d5a807-114b-4a8a-8228-f5e47a48f0c1 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:23.551318 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 645/1295] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Jul 27 04:12:23 2022] POST /identity/v3/services => generated 246 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:12:25.535416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f896c-f857-492f-ad13-fa237dee9baa None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:25.535943 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f896c-f857-492f-ad13-fa237dee9baa None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:25.536043 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f896c-f857-492f-ad13-fa237dee9baa None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:25.537325 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 651/1296] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:12:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:12:25.542333 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9e097-3365-4f47-954c-2f8f02e3e09a None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:25.542732 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9e097-3365-4f47-954c-2f8f02e3e09a None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:25.542945 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9e097-3365-4f47-954c-2f8f02e3e09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:25.561040 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-0fe9e097-3365-4f47-954c-2f8f02e3e09a None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:12:25.562249 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0fe9e097-3365-4f47-954c-2f8f02e3e09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:12:25.632688 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 646/1297] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:12:25 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:12:25.654874 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f896c-f857-492f-ad13-fa237dee9baa None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:25.755174 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f896c-f857-492f-ad13-fa237dee9baa None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:25.787488 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec02fcf8-e977-4021-8527-e6f3cbe851ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:25.788131 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02fcf8-e977-4021-8527-e6f3cbe851ce None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:25.788418 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02fcf8-e977-4021-8527-e6f3cbe851ce None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:25.788697 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02fcf8-e977-4021-8527-e6f3cbe851ce None admin] PATH_INFO: `/v3/services/placement` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:25.789331 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec02fcf8-e977-4021-8527-e6f3cbe851ce None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:25.790712 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec02fcf8-e977-4021-8527-e6f3cbe851ce None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:25.800134 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-ec02fcf8-e977-4021-8527-e6f3cbe851ce None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 04:12:25.800134 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:12:25.800134 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 04:12:25.800134 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 04:12:25.800134 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 04:12:25.800134 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 04:12:25.800134 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 04:12:25.800134 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:12:25.800134 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 04:12:25.800134 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:12:25.800134 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 04:12:25.800134 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:12:25.800134 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:12:25.800134 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:12:25.800134 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:12:25.800134 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:12:25.800134 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:12:25.801368 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:12:25.801368 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:12:25.801368 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:12:25.801368 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:12:25.801368 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:12:25.801368 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:12:25.801368 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 04:12:25.801368 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 04:12:25.801368 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 04:12:25.801368 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 04:12:25.801368 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:12:25.801368 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:12:25.801368 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:12:25.801368 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:12:25.801368 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:12:25.801368 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:12:25.802566 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:12:25.802566 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:12:25.802566 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:12:25.802566 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:12:25.802566 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:12:25.802566 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:12:25.802566 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:12:25.802566 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:12:25.802566 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:12:25.802566 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:12:25.802566 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 04:12:25.802566 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:12:25.802566 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 04:12:25.802566 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:12:25.802566 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 652/1298] 10.222.0.46 () {56 vars in 1117 bytes} [Wed Jul 27 04:12:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:12:25.807601 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9e097-3365-4f47-954c-2f8f02e3e09a None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:25.826165 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:25.904818 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9e097-3365-4f47-954c-2f8f02e3e09a None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:25.941514 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54bf1bc-9d75-400e-a17d-033a57d64618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:25.942165 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54bf1bc-9d75-400e-a17d-033a57d64618 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:25.942442 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54bf1bc-9d75-400e-a17d-033a57d64618 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:25.942713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54bf1bc-9d75-400e-a17d-033a57d64618 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:25.943392 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54bf1bc-9d75-400e-a17d-033a57d64618 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:12:25.943691 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54bf1bc-9d75-400e-a17d-033a57d64618 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:25.945053 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54bf1bc-9d75-400e-a17d-033a57d64618 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:25.950294 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 647/1299] 10.222.0.46 () {56 vars in 1116 bytes} [Wed Jul 27 04:12:25 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:12:25.956554 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec02fcf8-e977-4021-8527-e6f3cbe851ce None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:25.987813 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:26.054654 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec02fcf8-e977-4021-8527-e6f3cbe851ce None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:26.086880 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e358e0-0995-4900-8a73-08b750dcdf9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:26.087490 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e358e0-0995-4900-8a73-08b750dcdf9b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:26.087769 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e358e0-0995-4900-8a73-08b750dcdf9b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:26.088042 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e358e0-0995-4900-8a73-08b750dcdf9b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:26.088798 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e358e0-0995-4900-8a73-08b750dcdf9b None admin] RBAC: Adding query filter params (interface=public, service_id=14ffc810e4e04be28f50112c9249a89a, region_id=RegionOne) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:12:26.089105 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e358e0-0995-4900-8a73-08b750dcdf9b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:26.090607 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e358e0-0995-4900-8a73-08b750dcdf9b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:26.095407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 653/1300] 10.222.0.46 () {56 vars in 1251 bytes} [Wed Jul 27 04:12:25 2022] GET /identity/v3/endpoints?service_id=14ffc810e4e04be28f50112c9249a89a&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:12:26.101132 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54bf1bc-9d75-400e-a17d-033a57d64618 None admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:26.181358 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:26.200194 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54bf1bc-9d75-400e-a17d-033a57d64618 None admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:26.237698 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d974b3-7e47-4d2b-8032-ef91188caddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:26.238418 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d974b3-7e47-4d2b-8032-ef91188caddf None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:26.238638 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d974b3-7e47-4d2b-8032-ef91188caddf None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:26.238912 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d974b3-7e47-4d2b-8032-ef91188caddf None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:26.239605 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d974b3-7e47-4d2b-8032-ef91188caddf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:26.241013 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d974b3-7e47-4d2b-8032-ef91188caddf None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:26.248088 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 648/1301] 10.222.0.46 () {56 vars in 1087 bytes} [Wed Jul 27 04:12:26 2022] GET /identity/v3/services => generated 2191 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:12:28.213743 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2ec17-c06f-41ab-b085-958205996463 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:28.214318 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2ec17-c06f-41ab-b085-958205996463 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:28.214428 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2ec17-c06f-41ab-b085-958205996463 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:28.215894 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 654/1302] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:12:28.221554 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db1261-19e2-4eb2-9aa4-cd6515a384f2 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:28.222010 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db1261-19e2-4eb2-9aa4-cd6515a384f2 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:28.222120 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db1261-19e2-4eb2-9aa4-cd6515a384f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:28.240861 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-21db1261-19e2-4eb2-9aa4-cd6515a384f2 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:12:28.242082 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-21db1261-19e2-4eb2-9aa4-cd6515a384f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:12:28.307903 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 649/1303] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:12:28 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 04:12:28.330302 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2ec17-c06f-41ab-b085-958205996463 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:28.421982 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:28.428726 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2ec17-c06f-41ab-b085-958205996463 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:28.461353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc72496-4c3a-4cd9-9174-133a3ad52225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:28.462027 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc72496-4c3a-4cd9-9174-133a3ad52225 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:28.462314 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc72496-4c3a-4cd9-9174-133a3ad52225 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:28.462588 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc72496-4c3a-4cd9-9174-133a3ad52225 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:28.463235 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc72496-4c3a-4cd9-9174-133a3ad52225 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:28.465239 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc72496-4c3a-4cd9-9174-133a3ad52225 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:28.476046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-7fc72496-4c3a-4cd9-9174-133a3ad52225 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 04:12:28.476046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:12:28.476046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 04:12:28.476046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 04:12:28.476046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 04:12:28.476046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 04:12:28.476046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 04:12:28.476046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:12:28.476046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 04:12:28.476046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:12:28.476046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 04:12:28.476046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:12:28.476046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 04:12:28.476046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 04:12:28.476046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 04:12:28.476046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 04:12:28.476046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 04:12:28.477249 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 04:12:28.477249 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 04:12:28.477249 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 04:12:28.477249 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 04:12:28.477249 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 04:12:28.477249 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 04:12:28.477249 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 04:12:28.477249 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 04:12:28.477249 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 04:12:28.477249 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 04:12:28.477249 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 04:12:28.477249 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 04:12:28.477249 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 04:12:28.477249 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 04:12:28.477249 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 04:12:28.477249 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 04:12:28.478124 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 04:12:28.478124 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jul 27 04:12:28.478124 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 04:12:28.478124 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jul 27 04:12:28.478124 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 04:12:28.478124 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 04:12:28.478124 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 04:12:28.478124 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jul 27 04:12:28.478124 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 04:12:28.478124 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jul 27 04:12:28.478124 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 04:12:28.478124 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 04:12:28.478124 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 04:12:28.478124 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jul 27 04:12:28.478124 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 655/1304] 10.222.0.46 () {56 vars in 1117 bytes} [Wed Jul 27 04:12:28 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 04:12:28.483901 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db1261-19e2-4eb2-9aa4-cd6515a384f2 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:28.557049 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:28.579539 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db1261-19e2-4eb2-9aa4-cd6515a384f2 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:28.612343 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517babf-a543-4e5f-8de2-6f67910bae70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:28.612949 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517babf-a543-4e5f-8de2-6f67910bae70 None admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:28.613227 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517babf-a543-4e5f-8de2-6f67910bae70 None admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:28.613499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517babf-a543-4e5f-8de2-6f67910bae70 None admin] PATH_INFO: `/v3/services` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:28.614199 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1517babf-a543-4e5f-8de2-6f67910bae70 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:12:28.614502 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1517babf-a543-4e5f-8de2-6f67910bae70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:28.615853 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1517babf-a543-4e5f-8de2-6f67910bae70 None admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:28.620859 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 650/1305] 10.222.0.46 () {56 vars in 1116 bytes} [Wed Jul 27 04:12:28 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:12:28.627036 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc72496-4c3a-4cd9-9174-133a3ad52225 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:12:28.724516 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc72496-4c3a-4cd9-9174-133a3ad52225 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:12:28.757070 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d41c8a-2243-412f-9a4f-f8314bd5b2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:12:28.757673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d41c8a-2243-412f-9a4f-f8314bd5b2be None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:12:28.757955 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d41c8a-2243-412f-9a4f-f8314bd5b2be None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:12:28.758258 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d41c8a-2243-412f-9a4f-f8314bd5b2be None admin] PATH_INFO: `/v3/endpoints` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:12:28.758972 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d41c8a-2243-412f-9a4f-f8314bd5b2be None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:12:28.760327 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d41c8a-2243-412f-9a4f-f8314bd5b2be None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:12:28.779297 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 656/1306] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:12:28 2022] POST /identity/v3/endpoints => generated 331 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:12:38.617851 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=30 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:12:38.775076 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:13:31.469616 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87d6dd-8858-4257-a0d2-07dfd4bb7694 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:31.470554 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87d6dd-8858-4257-a0d2-07dfd4bb7694 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:31.470998 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87d6dd-8858-4257-a0d2-07dfd4bb7694 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:31.477950 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15310aff-8310-479f-a84f-7c9a97f30158 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:31.479016 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15310aff-8310-479f-a84f-7c9a97f30158 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:31.479942 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15310aff-8310-479f-a84f-7c9a97f30158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:31.483642 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 651/1307] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:13:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:13:31.621342 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-15310aff-8310-479f-a84f-7c9a97f30158 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:13:31.623679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-15310aff-8310-479f-a84f-7c9a97f30158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:13:31.803130 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 657/1308] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:13:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:13:31.844130 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c56bf0-6456-4e9a-8bcf-144a3a2c528f None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:31.844471 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c56bf0-6456-4e9a-8bcf-144a3a2c528f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:31.844997 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c56bf0-6456-4e9a-8bcf-144a3a2c528f None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:31.846880 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 652/1309] 10.222.0.46 () {54 vars in 965 bytes} [Wed Jul 27 04:13:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:13:31.852943 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce35ff0-e8fd-43fb-a632-55fda01677ea None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:31.853510 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce35ff0-e8fd-43fb-a632-55fda01677ea None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:31.854616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce35ff0-e8fd-43fb-a632-55fda01677ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:31.944849 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-9ce35ff0-e8fd-43fb-a632-55fda01677ea None None] MFA Rules not processed for user `b4847f3736d844d78a206b7eab5f5fcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:13:31.946568 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-9ce35ff0-e8fd-43fb-a632-55fda01677ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:13:31.974525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:13:32.043613 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 658/1310] 10.222.0.46 () {58 vars in 1062 bytes} [Wed Jul 27 04:13:31 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:13:32.052821 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c56bf0-6456-4e9a-8bcf-144a3a2c528f None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:13:32.054830 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-62c56bf0-6456-4e9a-8bcf-144a3a2c528f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:13:32.208462 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c56bf0-6456-4e9a-8bcf-144a3a2c528f None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:13:32.303909 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023f756-30e5-4490-8cee-27d442483965 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:13:32.304705 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023f756-30e5-4490-8cee-27d442483965 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:32.304993 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023f756-30e5-4490-8cee-27d442483965 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:32.305269 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023f756-30e5-4490-8cee-27d442483965 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:32.308287 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0023f756-30e5-4490-8cee-27d442483965 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:13:32.372064 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:13:32.435881 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0023f756-30e5-4490-8cee-27d442483965 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:13:32.437458 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0023f756-30e5-4490-8cee-27d442483965 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:13:32.459162 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 653/1311] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:13:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:13:34.859625 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b16105-c9b1-461a-a47a-42eb4420f5b5 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:34.860184 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b16105-c9b1-461a-a47a-42eb4420f5b5 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:34.860184 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b16105-c9b1-461a-a47a-42eb4420f5b5 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:34.861537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 659/1312] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 04:13:34.866657 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703c1d1-a9b4-40b8-88a5-6b146b1907e6 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:34.867080 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703c1d1-a9b4-40b8-88a5-6b146b1907e6 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:34.867210 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703c1d1-a9b4-40b8-88a5-6b146b1907e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:34.950471 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-3703c1d1-a9b4-40b8-88a5-6b146b1907e6 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:13:34.951661 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3703c1d1-a9b4-40b8-88a5-6b146b1907e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:13:35.050774 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 654/1313] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:13:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:13:35.090782 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b16105-c9b1-461a-a47a-42eb4420f5b5 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:13:35.092437 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e8b16105-c9b1-461a-a47a-42eb4420f5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:13:35.208477 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b16105-c9b1-461a-a47a-42eb4420f5b5 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:13:35.284622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:13:35.287371 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb9dd82-b3a5-4932-987f-9db53d10454a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:13:35.288080 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9dd82-b3a5-4932-987f-9db53d10454a service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:35.288367 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9dd82-b3a5-4932-987f-9db53d10454a service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:35.288716 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9dd82-b3a5-4932-987f-9db53d10454a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:35.291850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-1cb9dd82-b3a5-4932-987f-9db53d10454a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:13:35.403698 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb9dd82-b3a5-4932-987f-9db53d10454a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:13:35.405157 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb9dd82-b3a5-4932-987f-9db53d10454a service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:13:35.423431 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 660/1314] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:13:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:13:37.868014 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3144146-094b-4f49-ad91-7a1f1e017a44 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:37.868570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3144146-094b-4f49-ad91-7a1f1e017a44 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:37.868570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3144146-094b-4f49-ad91-7a1f1e017a44 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:37.870528 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 655/1315] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:13:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:13:37.875539 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb64c39-2dd3-48e3-9b19-be77151a9c5b None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:37.875971 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb64c39-2dd3-48e3-9b19-be77151a9c5b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:37.876124 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb64c39-2dd3-48e3-9b19-be77151a9c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:37.940443 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-edb64c39-2dd3-48e3-9b19-be77151a9c5b None None] MFA Rules not processed for user `706e785a4bca46e7ab7003c417e51cc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:13:37.941644 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-edb64c39-2dd3-48e3-9b19-be77151a9c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:13:37.991826 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:13:38.036379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 661/1316] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:13:37 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:13:38.091953 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea8c5f8-9412-4811-9e66-5f8287b1dd7d None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:38.092233 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea8c5f8-9412-4811-9e66-5f8287b1dd7d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:38.092507 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea8c5f8-9412-4811-9e66-5f8287b1dd7d None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:38.093883 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 656/1317] 10.222.0.46 () {54 vars in 965 bytes} [Wed Jul 27 04:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:13:38.099789 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3182b-0343-4193-8de2-103c458ada04 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:38.100072 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3182b-0343-4193-8de2-103c458ada04 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:38.100351 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3182b-0343-4193-8de2-103c458ada04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:38.177708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-9fa3182b-0343-4193-8de2-103c458ada04 None None] MFA Rules not processed for user `b4847f3736d844d78a206b7eab5f5fcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:13:38.178875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-9fa3182b-0343-4193-8de2-103c458ada04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:13:38.256578 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 662/1318] 10.222.0.46 () {58 vars in 1062 bytes} [Wed Jul 27 04:13:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:13:38.265671 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea8c5f8-9412-4811-9e66-5f8287b1dd7d None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:13:38.267234 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-aea8c5f8-9412-4811-9e66-5f8287b1dd7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:13:38.277788 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:13:38.370969 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea8c5f8-9412-4811-9e66-5f8287b1dd7d None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:13:38.442865 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22336f94-3334-41ee-9dee-bc724519cf49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:13:38.443411 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22336f94-3334-41ee-9dee-bc724519cf49 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:38.443691 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22336f94-3334-41ee-9dee-bc724519cf49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:38.443965 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22336f94-3334-41ee-9dee-bc724519cf49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:38.445569 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-22336f94-3334-41ee-9dee-bc724519cf49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:13:38.508439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:13:38.558239 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22336f94-3334-41ee-9dee-bc724519cf49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:13:38.560204 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22336f94-3334-41ee-9dee-bc724519cf49 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:13:38.579562 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 657/1319] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:13:38 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:13:41.550526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532940f3-2a50-473b-ad7e-35a33e712071 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:41.551045 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532940f3-2a50-473b-ad7e-35a33e712071 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:41.551045 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532940f3-2a50-473b-ad7e-35a33e712071 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:41.552424 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 663/1320] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:13:41.557754 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c794f6b-85f0-4b15-b6e1-5f73345c63a3 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:41.558173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c794f6b-85f0-4b15-b6e1-5f73345c63a3 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:41.558361 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c794f6b-85f0-4b15-b6e1-5f73345c63a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:41.623741 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-7c794f6b-85f0-4b15-b6e1-5f73345c63a3 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:13:41.624923 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-7c794f6b-85f0-4b15-b6e1-5f73345c63a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:13:41.718660 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 658/1321] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:13:41 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:13:41.756694 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532940f3-2a50-473b-ad7e-35a33e712071 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:13:41.758223 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-532940f3-2a50-473b-ad7e-35a33e712071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:13:41.806675 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:13:41.859447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532940f3-2a50-473b-ad7e-35a33e712071 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:13:41.930049 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245caddc-fd55-4085-84a8-e20b9d8087ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:13:41.930695 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245caddc-fd55-4085-84a8-e20b9d8087ce service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:41.930979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245caddc-fd55-4085-84a8-e20b9d8087ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:41.931798 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245caddc-fd55-4085-84a8-e20b9d8087ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:41.933450 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-245caddc-fd55-4085-84a8-e20b9d8087ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:13:42.025744 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:13:42.039387 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245caddc-fd55-4085-84a8-e20b9d8087ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:13:42.041394 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245caddc-fd55-4085-84a8-e20b9d8087ce service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:13:42.059080 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 664/1322] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:13:41 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:13:44.745200 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874eb74b-c4c4-454a-8eea-63524ab50c97 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:44.745726 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874eb74b-c4c4-454a-8eea-63524ab50c97 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:44.745726 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874eb74b-c4c4-454a-8eea-63524ab50c97 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:44.747110 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 659/1323] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:13:44.752429 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f263fc5-cade-46d5-9821-240a2e2fc8e4 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:44.752881 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f263fc5-cade-46d5-9821-240a2e2fc8e4 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:44.753068 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f263fc5-cade-46d5-9821-240a2e2fc8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:44.819021 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-7f263fc5-cade-46d5-9821-240a2e2fc8e4 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:13:44.820248 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7f263fc5-cade-46d5-9821-240a2e2fc8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:13:44.921189 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 665/1324] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:13:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:13:44.958109 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874eb74b-c4c4-454a-8eea-63524ab50c97 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:13:44.959689 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-874eb74b-c4c4-454a-8eea-63524ab50c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:13:44.980961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:13:45.072541 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874eb74b-c4c4-454a-8eea-63524ab50c97 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:13:45.146451 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe1ed9-5f16-4a26-9b07-fd5a1a6802ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:13:45.147068 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe1ed9-5f16-4a26-9b07-fd5a1a6802ca service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:45.147343 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe1ed9-5f16-4a26-9b07-fd5a1a6802ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:45.147609 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe1ed9-5f16-4a26-9b07-fd5a1a6802ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:45.149214 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9dbe1ed9-5f16-4a26-9b07-fd5a1a6802ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:13:45.222893 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:13:45.262667 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe1ed9-5f16-4a26-9b07-fd5a1a6802ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:13:45.264093 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe1ed9-5f16-4a26-9b07-fd5a1a6802ca service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:13:45.282715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 660/1325] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:13:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:13:47.350466 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b07de-c40f-4643-aed0-a0ec77defd0c None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:47.351022 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b07de-c40f-4643-aed0-a0ec77defd0c None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:47.351022 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b07de-c40f-4643-aed0-a0ec77defd0c None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:47.352372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 666/1326] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:13:47.358391 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377b116-2701-4e25-944d-f0e16c268e57 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:47.358800 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377b116-2701-4e25-944d-f0e16c268e57 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:47.358943 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377b116-2701-4e25-944d-f0e16c268e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:47.423509 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-c377b116-2701-4e25-944d-f0e16c268e57 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:13:47.424588 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c377b116-2701-4e25-944d-f0e16c268e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:13:47.523084 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 661/1327] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:13:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:13:47.560126 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726b07de-c40f-4643-aed0-a0ec77defd0c None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:13:47.561658 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-726b07de-c40f-4643-aed0-a0ec77defd0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:13:47.622627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:13:47.664867 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726b07de-c40f-4643-aed0-a0ec77defd0c None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:13:47.737712 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b5b10-f93e-4453-bfab-48fb32dd2f4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:13:47.738281 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b5b10-f93e-4453-bfab-48fb32dd2f4c service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:47.738675 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b5b10-f93e-4453-bfab-48fb32dd2f4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:47.738956 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b5b10-f93e-4453-bfab-48fb32dd2f4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:47.740546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-da4b5b10-f93e-4453-bfab-48fb32dd2f4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:13:47.851562 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:13:47.854914 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b5b10-f93e-4453-bfab-48fb32dd2f4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:13:47.856941 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b5b10-f93e-4453-bfab-48fb32dd2f4c service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:13:47.876551 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 667/1328] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:13:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:13:49.917343 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b339f5-e711-49c6-b7c9-c0a403e30421 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:49.917844 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b339f5-e711-49c6-b7c9-c0a403e30421 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:49.917925 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b339f5-e711-49c6-b7c9-c0a403e30421 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:49.919310 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 662/1329] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:13:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:13:49.924457 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10de52e-9023-4105-977d-f4e1ef96ec7f None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:49.924855 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10de52e-9023-4105-977d-f4e1ef96ec7f None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:49.925095 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10de52e-9023-4105-977d-f4e1ef96ec7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:49.997131 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-f10de52e-9023-4105-977d-f4e1ef96ec7f None None] MFA Rules not processed for user `706e785a4bca46e7ab7003c417e51cc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:13:49.998357 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f10de52e-9023-4105-977d-f4e1ef96ec7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:13:50.092714 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 668/1330] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:13:49 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:13:50.130440 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b339f5-e711-49c6-b7c9-c0a403e30421 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:13:50.131954 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-58b339f5-e711-49c6-b7c9-c0a403e30421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:13:50.164840 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:13:50.240123 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b339f5-e711-49c6-b7c9-c0a403e30421 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:13:50.321181 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701d192-c387-4d40-8342-98b7cdf379b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:13:50.322456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701d192-c387-4d40-8342-98b7cdf379b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:50.322741 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701d192-c387-4d40-8342-98b7cdf379b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:50.323015 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701d192-c387-4d40-8342-98b7cdf379b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:50.324646 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-a701d192-c387-4d40-8342-98b7cdf379b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:13:50.405740 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:13:50.439735 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a701d192-c387-4d40-8342-98b7cdf379b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:13:50.441128 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a701d192-c387-4d40-8342-98b7cdf379b7 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:13:50.459854 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 663/1331] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:13:50 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:13:52.743953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b97185-a189-4abd-9377-e19e8a72b87d None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:52.744452 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b97185-a189-4abd-9377-e19e8a72b87d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:52.744452 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b97185-a189-4abd-9377-e19e8a72b87d None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:52.745815 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 669/1332] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:13:52.751020 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d494d89-5032-426b-b6b2-aa9c57c74301 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:52.751445 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d494d89-5032-426b-b6b2-aa9c57c74301 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:52.751551 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d494d89-5032-426b-b6b2-aa9c57c74301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:52.816141 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-4d494d89-5032-426b-b6b2-aa9c57c74301 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:13:52.817340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-4d494d89-5032-426b-b6b2-aa9c57c74301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:13:52.915116 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 664/1333] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:13:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:13:52.951777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b97185-a189-4abd-9377-e19e8a72b87d None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:13:52.953297 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f1b97185-a189-4abd-9377-e19e8a72b87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:13:53.020883 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:13:53.055249 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b97185-a189-4abd-9377-e19e8a72b87d None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:13:53.127503 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708862c9-fd3d-4aa7-a7a1-f97c44c6843d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:13:53.128118 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708862c9-fd3d-4aa7-a7a1-f97c44c6843d service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:53.128396 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708862c9-fd3d-4aa7-a7a1-f97c44c6843d service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:53.128662 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708862c9-fd3d-4aa7-a7a1-f97c44c6843d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:53.130285 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-708862c9-fd3d-4aa7-a7a1-f97c44c6843d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:13:53.236562 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708862c9-fd3d-4aa7-a7a1-f97c44c6843d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:13:53.237999 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708862c9-fd3d-4aa7-a7a1-f97c44c6843d service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:13:53.248484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:13:53.256795 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 670/1334] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:13:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:13:55.303383 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd6874-e258-463f-99f3-1bc447febe0f None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:55.303993 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd6874-e258-463f-99f3-1bc447febe0f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:55.303993 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd6874-e258-463f-99f3-1bc447febe0f None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:55.305285 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 665/1335] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:13:55.310532 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21821b89-b895-4825-b34f-d8cca4a85e95 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:55.310969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21821b89-b895-4825-b34f-d8cca4a85e95 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:55.311081 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21821b89-b895-4825-b34f-d8cca4a85e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:55.378126 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-21821b89-b895-4825-b34f-d8cca4a85e95 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:13:55.379324 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-21821b89-b895-4825-b34f-d8cca4a85e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:13:55.474308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 671/1336] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:13:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:13:55.515272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd6874-e258-463f-99f3-1bc447febe0f None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:13:55.516797 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-a1fd6874-e258-463f-99f3-1bc447febe0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:13:55.560063 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:13:55.624135 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd6874-e258-463f-99f3-1bc447febe0f None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:13:55.697270 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7515b960-fcd4-40eb-8d7d-c0f009b505e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:13:55.698172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7515b960-fcd4-40eb-8d7d-c0f009b505e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:55.698457 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7515b960-fcd4-40eb-8d7d-c0f009b505e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:55.698732 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7515b960-fcd4-40eb-8d7d-c0f009b505e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:55.700339 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-7515b960-fcd4-40eb-8d7d-c0f009b505e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:13:55.783834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:13:55.811427 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7515b960-fcd4-40eb-8d7d-c0f009b505e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:13:55.812796 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7515b960-fcd4-40eb-8d7d-c0f009b505e4 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:13:55.830747 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 666/1337] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:13:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:13:58.194520 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0ac5b-32d5-4790-831c-f641890435fb None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:58.195080 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0ac5b-32d5-4790-831c-f641890435fb None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:58.195080 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0ac5b-32d5-4790-831c-f641890435fb None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:58.196959 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 672/1338] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:13:58.202320 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e0079-6b2b-42b4-9066-7113344f80e9 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:58.202842 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e0079-6b2b-42b4-9066-7113344f80e9 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:58.202842 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e0079-6b2b-42b4-9066-7113344f80e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:58.268767 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-1d2e0079-6b2b-42b4-9066-7113344f80e9 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:13:58.270010 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-1d2e0079-6b2b-42b4-9066-7113344f80e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:13:58.374983 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 667/1339] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:13:58 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:13:58.412484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f0ac5b-32d5-4790-831c-f641890435fb None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:13:58.414036 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-57f0ac5b-32d5-4790-831c-f641890435fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:13:58.490121 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:13:58.519378 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f0ac5b-32d5-4790-831c-f641890435fb None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:13:58.592249 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef34db-3deb-4dc9-ac58-bc9f1909cf80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:13:58.592875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef34db-3deb-4dc9-ac58-bc9f1909cf80 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:13:58.593159 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef34db-3deb-4dc9-ac58-bc9f1909cf80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:13:58.593437 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef34db-3deb-4dc9-ac58-bc9f1909cf80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:13:58.595092 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-bcef34db-3deb-4dc9-ac58-bc9f1909cf80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:13:58.701757 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcef34db-3deb-4dc9-ac58-bc9f1909cf80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:13:58.703229 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcef34db-3deb-4dc9-ac58-bc9f1909cf80 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:13:58.720706 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 673/1340] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:13:58 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:14:02.657818 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395e25a-ca1b-475e-81fe-e679879cd2fd None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:02.658397 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395e25a-ca1b-475e-81fe-e679879cd2fd None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:02.658645 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395e25a-ca1b-475e-81fe-e679879cd2fd None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:02.660147 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 668/1341] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:14:02.665556 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd8e53-3d79-4f8c-a494-298098925b43 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:02.665949 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd8e53-3d79-4f8c-a494-298098925b43 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:02.666074 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd8e53-3d79-4f8c-a494-298098925b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:02.671551 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:02.733883 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-55fd8e53-3d79-4f8c-a494-298098925b43 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:14:02.735114 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-55fd8e53-3d79-4f8c-a494-298098925b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:02.830775 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 674/1342] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:14:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:14:02.871288 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395e25a-ca1b-475e-81fe-e679879cd2fd None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:14:02.872823 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-4395e25a-ca1b-475e-81fe-e679879cd2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:02.922440 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:02.980047 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395e25a-ca1b-475e-81fe-e679879cd2fd None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:14:03.054140 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bfdd02-6f09-4469-94e8-aa20dceb8844 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:14:03.054767 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfdd02-6f09-4469-94e8-aa20dceb8844 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:03.055045 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfdd02-6f09-4469-94e8-aa20dceb8844 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:03.055318 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfdd02-6f09-4469-94e8-aa20dceb8844 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:03.057344 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-79bfdd02-6f09-4469-94e8-aa20dceb8844 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:03.152437 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:03.170324 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bfdd02-6f09-4469-94e8-aa20dceb8844 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:14:03.172383 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bfdd02-6f09-4469-94e8-aa20dceb8844 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:14:03.196951 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 669/1343] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:14:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:14:05.840550 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc68c78-c7cc-4cd4-9682-e3c941b55b66 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:05.841126 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc68c78-c7cc-4cd4-9682-e3c941b55b66 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:05.841126 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc68c78-c7cc-4cd4-9682-e3c941b55b66 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:05.842625 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 675/1344] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:14:05.848514 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b1bdd-7ea1-460d-9fda-1d445d7ba588 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:05.849032 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b1bdd-7ea1-460d-9fda-1d445d7ba588 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:05.849196 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b1bdd-7ea1-460d-9fda-1d445d7ba588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:05.923422 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-2d0b1bdd-7ea1-460d-9fda-1d445d7ba588 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:14:05.924640 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2d0b1bdd-7ea1-460d-9fda-1d445d7ba588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:06.029311 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 670/1345] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:14:05 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:14:06.067056 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc68c78-c7cc-4cd4-9682-e3c941b55b66 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:14:06.068672 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-adc68c78-c7cc-4cd4-9682-e3c941b55b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:06.157034 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:06.174215 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc68c78-c7cc-4cd4-9682-e3c941b55b66 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:14:06.253659 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf4cdc1-d7f9-4ab4-be94-ab3017876d9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:14:06.254233 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4cdc1-d7f9-4ab4-be94-ab3017876d9d service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:06.254520 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4cdc1-d7f9-4ab4-be94-ab3017876d9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:06.254794 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4cdc1-d7f9-4ab4-be94-ab3017876d9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:06.256411 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-fcf4cdc1-d7f9-4ab4-be94-ab3017876d9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:06.365489 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf4cdc1-d7f9-4ab4-be94-ab3017876d9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:14:06.366956 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf4cdc1-d7f9-4ab4-be94-ab3017876d9d service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:14:06.385659 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 676/1346] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:14:06 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:14:10.418469 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6014f1-fd77-431d-8dd5-62467670a131 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:10.419031 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6014f1-fd77-431d-8dd5-62467670a131 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:10.419031 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6014f1-fd77-431d-8dd5-62467670a131 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:10.420381 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 671/1347] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:14:10.426135 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6f41b-c334-4552-bd4b-ae86796424b0 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:10.426578 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6f41b-c334-4552-bd4b-ae86796424b0 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:10.426698 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6f41b-c334-4552-bd4b-ae86796424b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:10.444909 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:10.507183 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-b9d6f41b-c334-4552-bd4b-ae86796424b0 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:14:10.508461 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b9d6f41b-c334-4552-bd4b-ae86796424b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:10.606610 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 677/1348] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:14:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:14:10.651201 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6014f1-fd77-431d-8dd5-62467670a131 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:14:10.652737 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ce6014f1-fd77-431d-8dd5-62467670a131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:10.732309 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:10.783406 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6014f1-fd77-431d-8dd5-62467670a131 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:14:10.871923 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8905441-db18-46be-9e68-d925abf1f5a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:14:10.872548 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8905441-db18-46be-9e68-d925abf1f5a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:10.872821 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8905441-db18-46be-9e68-d925abf1f5a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:10.873089 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8905441-db18-46be-9e68-d925abf1f5a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:10.874751 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d8905441-db18-46be-9e68-d925abf1f5a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:10.987758 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:10.992051 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8905441-db18-46be-9e68-d925abf1f5a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:14:10.993699 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8905441-db18-46be-9e68-d925abf1f5a7 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:14:11.013451 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 672/1349] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:14:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:14:15.933392 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae14bc7-8b30-43f9-bf9e-2956548e9972 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:15.934293 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae14bc7-8b30-43f9-bf9e-2956548e9972 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:15.935707 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae14bc7-8b30-43f9-bf9e-2956548e9972 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:15.937598 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 678/1350] 10.222.0.46 () {54 vars in 921 bytes} [Wed Jul 27 04:14:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:14:15.944683 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb36f47-0163-4990-9f19-04d9c2d48d73 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:15.945497 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb36f47-0163-4990-9f19-04d9c2d48d73 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:15.946076 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb36f47-0163-4990-9f19-04d9c2d48d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:16.100329 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-7bb36f47-0163-4990-9f19-04d9c2d48d73 None None] MFA Rules not processed for user `8c08fba2d7de42218400c5e652cc3de0`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:14:16.102887 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-7bb36f47-0163-4990-9f19-04d9c2d48d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:16.224418 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 673/1351] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Jul 27 04:14:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:14:20.310986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311caf1-c4df-44dd-86ad-6bae80fe4835 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:20.311521 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311caf1-c4df-44dd-86ad-6bae80fe4835 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:20.311521 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311caf1-c4df-44dd-86ad-6bae80fe4835 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:20.313101 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 679/1352] 10.222.0.46 () {54 vars in 921 bytes} [Wed Jul 27 04:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:14:20.318357 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f68f9-e23c-44a1-93bf-82a148ce6ac2 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:20.319158 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f68f9-e23c-44a1-93bf-82a148ce6ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:20.319497 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f68f9-e23c-44a1-93bf-82a148ce6ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:20.375431 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:20.401320 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-927f68f9-e23c-44a1-93bf-82a148ce6ac2 None None] MFA Rules not processed for user `8c08fba2d7de42218400c5e652cc3de0`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:14:20.402552 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-927f68f9-e23c-44a1-93bf-82a148ce6ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:20.524036 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 674/1353] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Jul 27 04:14:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:14:20.604759 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=59 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:22.007439 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79852d8-a583-4af1-b0ed-f07f1e406b4b None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:22.008051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79852d8-a583-4af1-b0ed-f07f1e406b4b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:22.008051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79852d8-a583-4af1-b0ed-f07f1e406b4b None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:22.010121 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 680/1354] 10.222.0.46 () {54 vars in 921 bytes} [Wed Jul 27 04:14:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:14:22.015271 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ba945-7613-4c19-83a2-0a0e7b36771f None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:22.015679 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ba945-7613-4c19-83a2-0a0e7b36771f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:22.015853 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ba945-7613-4c19-83a2-0a0e7b36771f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:22.093831 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-195ba945-7613-4c19-83a2-0a0e7b36771f None None] MFA Rules not processed for user `8c08fba2d7de42218400c5e652cc3de0`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:14:22.095057 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-195ba945-7613-4c19-83a2-0a0e7b36771f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:22.138780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:22.206171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 675/1355] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Jul 27 04:14:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:14:25.635567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ab625-3ce8-4fd4-b19d-ac14ce62e491 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:25.637480 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ab625-3ce8-4fd4-b19d-ac14ce62e491 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:25.637480 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ab625-3ce8-4fd4-b19d-ac14ce62e491 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:25.642493 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 681/1356] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:14:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:14:25.644722 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390546d3-9a6c-47c6-ae89-66e4cc5e4a9b None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:25.645563 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390546d3-9a6c-47c6-ae89-66e4cc5e4a9b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:25.646116 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390546d3-9a6c-47c6-ae89-66e4cc5e4a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:25.722874 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-390546d3-9a6c-47c6-ae89-66e4cc5e4a9b None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:14:25.724105 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-390546d3-9a6c-47c6-ae89-66e4cc5e4a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:25.827665 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 676/1357] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:14:25.884361 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598280c-c4be-4936-baf2-95467b5b9216 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:25.884648 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598280c-c4be-4936-baf2-95467b5b9216 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:25.884954 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598280c-c4be-4936-baf2-95467b5b9216 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:25.886495 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 682/1358] 10.222.0.46 () {54 vars in 962 bytes} [Wed Jul 27 04:14:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:14:25.898232 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44211cf3-e92a-48ab-a439-5c10ea3e202a None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:25.898958 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44211cf3-e92a-48ab-a439-5c10ea3e202a None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:25.900150 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44211cf3-e92a-48ab-a439-5c10ea3e202a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:25.921862 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:25.986905 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-44211cf3-e92a-48ab-a439-5c10ea3e202a None None] MFA Rules not processed for user `9dc9ea8f89c14a2a8e3272f750647dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:14:25.988146 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-44211cf3-e92a-48ab-a439-5c10ea3e202a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:26.124677 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 677/1359] 10.222.0.46 () {58 vars in 1059 bytes} [Wed Jul 27 04:14:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:14:26.132669 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598280c-c4be-4936-baf2-95467b5b9216 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:14:26.134425 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2598280c-c4be-4936-baf2-95467b5b9216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:26.303164 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598280c-c4be-4936-baf2-95467b5b9216 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:14:26.379417 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:26.422547 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b720208-407f-4c40-8d1d-499ec1bccf24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:14:26.423178 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b720208-407f-4c40-8d1d-499ec1bccf24 service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:26.423459 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b720208-407f-4c40-8d1d-499ec1bccf24 service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:26.423732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b720208-407f-4c40-8d1d-499ec1bccf24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:26.425352 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-9b720208-407f-4c40-8d1d-499ec1bccf24 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:26.542683 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b720208-407f-4c40-8d1d-499ec1bccf24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:14:26.544250 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b720208-407f-4c40-8d1d-499ec1bccf24 service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:14:26.562926 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 683/1360] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:14:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 04:14:28.805131 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a9bf3-9af2-4283-8e8a-59197d220d1e None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:28.806539 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a9bf3-9af2-4283-8e8a-59197d220d1e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:28.806539 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a9bf3-9af2-4283-8e8a-59197d220d1e None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:28.807865 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 678/1361] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:14:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:14:28.812914 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3185d2a-4727-4376-a032-8d407e441173 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:28.813745 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3185d2a-4727-4376-a032-8d407e441173 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:28.814062 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3185d2a-4727-4376-a032-8d407e441173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:28.886048 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-f3185d2a-4727-4376-a032-8d407e441173 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:14:28.887262 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f3185d2a-4727-4376-a032-8d407e441173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:28.927523 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:29.005496 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 684/1362] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:14:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:14:29.049602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a9bf3-9af2-4283-8e8a-59197d220d1e None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:14:29.052636 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-bb0a9bf3-9af2-4283-8e8a-59197d220d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:29.125674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:29.249897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a9bf3-9af2-4283-8e8a-59197d220d1e None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:14:29.369859 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9449d0-b8e1-4eb4-bcd8-bbef0dac717b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:14:29.370517 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9449d0-b8e1-4eb4-bcd8-bbef0dac717b service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:29.370801 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9449d0-b8e1-4eb4-bcd8-bbef0dac717b service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:29.371080 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9449d0-b8e1-4eb4-bcd8-bbef0dac717b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:29.372738 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-dc9449d0-b8e1-4eb4-bcd8-bbef0dac717b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:29.397419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:29.475773 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9449d0-b8e1-4eb4-bcd8-bbef0dac717b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:14:29.478197 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9449d0-b8e1-4eb4-bcd8-bbef0dac717b service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:14:29.498201 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 679/1363] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:14:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:14:31.889606 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7d0e4-4947-4a31-8fdd-ac42f3ca07e6 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:31.890200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7d0e4-4947-4a31-8fdd-ac42f3ca07e6 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:31.890200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7d0e4-4947-4a31-8fdd-ac42f3ca07e6 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:31.891550 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 685/1364] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:14:31.896982 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06cbd5-c13c-426c-ab7e-75c54727d7c2 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:31.897428 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06cbd5-c13c-426c-ab7e-75c54727d7c2 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:31.897511 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06cbd5-c13c-426c-ab7e-75c54727d7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:31.967281 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-4a06cbd5-c13c-426c-ab7e-75c54727d7c2 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:14:31.968487 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-4a06cbd5-c13c-426c-ab7e-75c54727d7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:32.068624 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:32.074024 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 680/1365] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:14:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:14:32.118594 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983690a-5a59-4402-8b38-67c03e7bde2d None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:32.119412 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983690a-5a59-4402-8b38-67c03e7bde2d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:32.119929 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983690a-5a59-4402-8b38-67c03e7bde2d None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:32.121381 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 686/1366] 10.222.0.46 () {54 vars in 962 bytes} [Wed Jul 27 04:14:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:14:32.127066 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae05809-b3f4-4d3e-bd53-633f62e1bbd5 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:32.127358 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae05809-b3f4-4d3e-bd53-633f62e1bbd5 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:32.127635 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae05809-b3f4-4d3e-bd53-633f62e1bbd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:32.207236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-fae05809-b3f4-4d3e-bd53-633f62e1bbd5 None None] MFA Rules not processed for user `9dc9ea8f89c14a2a8e3272f750647dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:14:32.208439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-fae05809-b3f4-4d3e-bd53-633f62e1bbd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:32.336558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 681/1367] 10.222.0.46 () {58 vars in 1059 bytes} [Wed Jul 27 04:14:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:14:32.345119 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983690a-5a59-4402-8b38-67c03e7bde2d None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:14:32.346832 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c983690a-5a59-4402-8b38-67c03e7bde2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:32.517047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983690a-5a59-4402-8b38-67c03e7bde2d None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:14:32.538066 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:32.639855 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9773bd0-ddcc-4ead-8583-7b6c89849832 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:14:32.640397 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9773bd0-ddcc-4ead-8583-7b6c89849832 service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:32.640684 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9773bd0-ddcc-4ead-8583-7b6c89849832 service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:32.640966 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9773bd0-ddcc-4ead-8583-7b6c89849832 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:32.642619 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d9773bd0-ddcc-4ead-8583-7b6c89849832 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:32.739110 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9773bd0-ddcc-4ead-8583-7b6c89849832 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:14:32.740710 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9773bd0-ddcc-4ead-8583-7b6c89849832 service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:14:32.763541 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 687/1368] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:14:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:14:33.683815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4635d9-c525-4689-bc1f-e5157cf0bf19 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:33.684604 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4635d9-c525-4689-bc1f-e5157cf0bf19 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:33.684604 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4635d9-c525-4689-bc1f-e5157cf0bf19 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:33.686185 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 682/1369] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:14:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:14:33.692694 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4281bee-c0bc-4a3d-b29d-df3ef1c1b45b None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:33.693119 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4281bee-c0bc-4a3d-b29d-df3ef1c1b45b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:33.693342 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4281bee-c0bc-4a3d-b29d-df3ef1c1b45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:33.726161 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:33.763618 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-a4281bee-c0bc-4a3d-b29d-df3ef1c1b45b None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:14:33.764821 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a4281bee-c0bc-4a3d-b29d-df3ef1c1b45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:33.886637 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 688/1370] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:14:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:14:34.247116 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b0f33-aa25-46e1-aa0e-182bd1a2527d None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:34.252844 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b0f33-aa25-46e1-aa0e-182bd1a2527d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:34.252844 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b0f33-aa25-46e1-aa0e-182bd1a2527d None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:34.252844 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 683/1371] 10.222.0.46 () {54 vars in 964 bytes} [Wed Jul 27 04:14:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:14:34.257605 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427071c9-ebe5-46a2-aeb2-b4309141ee25 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:34.257959 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427071c9-ebe5-46a2-aeb2-b4309141ee25 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:34.258334 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427071c9-ebe5-46a2-aeb2-b4309141ee25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:34.340303 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-427071c9-ebe5-46a2-aeb2-b4309141ee25 None None] MFA Rules not processed for user `134ef452d7814c73b7c371a1cdbcbc00`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:14:34.341737 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-427071c9-ebe5-46a2-aeb2-b4309141ee25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:34.390091 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:34.421813 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 689/1372] 10.222.0.46 () {58 vars in 1061 bytes} [Wed Jul 27 04:14:34 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:14:34.429461 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b0f33-aa25-46e1-aa0e-182bd1a2527d None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:14:34.431599 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-310b0f33-aa25-46e1-aa0e-182bd1a2527d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:34.491577 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:34.548565 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b0f33-aa25-46e1-aa0e-182bd1a2527d None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:14:34.626048 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38eb3a-8a69-4fa5-abc6-ca50242f0b41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:14:34.626766 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38eb3a-8a69-4fa5-abc6-ca50242f0b41 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:34.627110 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38eb3a-8a69-4fa5-abc6-ca50242f0b41 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:34.627449 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38eb3a-8a69-4fa5-abc6-ca50242f0b41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:34.629119 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ec38eb3a-8a69-4fa5-abc6-ca50242f0b41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:34.745291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:34.763099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec38eb3a-8a69-4fa5-abc6-ca50242f0b41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:14:34.764825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec38eb3a-8a69-4fa5-abc6-ca50242f0b41 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:14:34.785464 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 684/1373] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:14:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:14:34.869289 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557fd0d8-7f95-4bdb-91e8-e3426c9ccbc2 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:34.869289 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557fd0d8-7f95-4bdb-91e8-e3426c9ccbc2 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:34.869289 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557fd0d8-7f95-4bdb-91e8-e3426c9ccbc2 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:34.874254 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 690/1374] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:14:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:14:34.875095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f69ec0-57ca-4d1a-bf21-55de6b382ca3 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:34.875410 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f69ec0-57ca-4d1a-bf21-55de6b382ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:34.875670 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f69ec0-57ca-4d1a-bf21-55de6b382ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:34.967349 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-61f69ec0-57ca-4d1a-bf21-55de6b382ca3 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:14:34.969048 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-61f69ec0-57ca-4d1a-bf21-55de6b382ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:35.095528 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 685/1375] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:14:35.140113 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557fd0d8-7f95-4bdb-91e8-e3426c9ccbc2 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:14:35.141623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-557fd0d8-7f95-4bdb-91e8-e3426c9ccbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:35.288437 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557fd0d8-7f95-4bdb-91e8-e3426c9ccbc2 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:14:35.340276 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:35.401558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b4684-52fb-49e2-971a-7998bdab3497 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:14:35.402208 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b4684-52fb-49e2-971a-7998bdab3497 service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:35.402494 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b4684-52fb-49e2-971a-7998bdab3497 service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:35.402768 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b4684-52fb-49e2-971a-7998bdab3497 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:35.404391 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-366b4684-52fb-49e2-971a-7998bdab3497 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:35.502741 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366b4684-52fb-49e2-971a-7998bdab3497 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:14:35.504351 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366b4684-52fb-49e2-971a-7998bdab3497 service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:14:35.523618 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 691/1376] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:14:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:14:37.592979 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1cfba-3e86-4343-b53f-21d1fa557179 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:37.593510 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1cfba-3e86-4343-b53f-21d1fa557179 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:37.593510 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1cfba-3e86-4343-b53f-21d1fa557179 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:37.594926 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 686/1377] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:14:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:14:37.600173 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07e3f2-b84a-4fde-bc88-8f5b43ce4a74 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:37.600627 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07e3f2-b84a-4fde-bc88-8f5b43ce4a74 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:37.600768 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07e3f2-b84a-4fde-bc88-8f5b43ce4a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:37.668591 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-4a07e3f2-b84a-4fde-bc88-8f5b43ce4a74 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:14:37.669888 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-4a07e3f2-b84a-4fde-bc88-8f5b43ce4a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:37.676447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:37.780581 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 692/1378] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:14:37.829156 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce1cfba-3e86-4343-b53f-21d1fa557179 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:14:37.830904 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-bce1cfba-3e86-4343-b53f-21d1fa557179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:37.897947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:37.984375 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce1cfba-3e86-4343-b53f-21d1fa557179 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:14:38.106772 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a91a1d8-f629-4514-b3e9-2b1570bcd16c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:14:38.107463 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91a1d8-f629-4514-b3e9-2b1570bcd16c service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:38.107750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91a1d8-f629-4514-b3e9-2b1570bcd16c service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:38.108025 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91a1d8-f629-4514-b3e9-2b1570bcd16c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:38.109673 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2a91a1d8-f629-4514-b3e9-2b1570bcd16c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:38.128442 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:38.215596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a91a1d8-f629-4514-b3e9-2b1570bcd16c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:14:38.217366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a91a1d8-f629-4514-b3e9-2b1570bcd16c service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:14:38.245247 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 687/1379] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:14:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:14:40.376362 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56135208-ad2f-49ab-9999-7ffa8e204b83 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:40.376886 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56135208-ad2f-49ab-9999-7ffa8e204b83 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:40.376886 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56135208-ad2f-49ab-9999-7ffa8e204b83 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:40.378447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 693/1380] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:14:40.383974 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa865c3-f133-4524-a03c-7d79cd13476a None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:40.385030 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa865c3-f133-4524-a03c-7d79cd13476a None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:40.385030 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa865c3-f133-4524-a03c-7d79cd13476a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:40.468887 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-8aa865c3-f133-4524-a03c-7d79cd13476a None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:14:40.470107 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-8aa865c3-f133-4524-a03c-7d79cd13476a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:40.573022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:40.585125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 688/1381] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:14:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:14:40.624512 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56135208-ad2f-49ab-9999-7ffa8e204b83 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:14:40.626931 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-56135208-ad2f-49ab-9999-7ffa8e204b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:40.773603 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:40.785867 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56135208-ad2f-49ab-9999-7ffa8e204b83 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:14:40.911993 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4628baf-f8cf-4943-bcc8-1debae4e25d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:14:40.912625 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4628baf-f8cf-4943-bcc8-1debae4e25d9 service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:40.912904 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4628baf-f8cf-4943-bcc8-1debae4e25d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:40.913181 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4628baf-f8cf-4943-bcc8-1debae4e25d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:40.915426 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d4628baf-f8cf-4943-bcc8-1debae4e25d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:41.019726 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4628baf-f8cf-4943-bcc8-1debae4e25d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:14:41.021439 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4628baf-f8cf-4943-bcc8-1debae4e25d9 service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:14:41.031841 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:41.041150 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 694/1382] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:14:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:14:43.318993 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77e50d-e62b-4dc9-96eb-5581e27454d6 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:43.319533 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77e50d-e62b-4dc9-96eb-5581e27454d6 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:43.319533 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77e50d-e62b-4dc9-96eb-5581e27454d6 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:43.320959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 689/1383] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:14:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:14:43.327664 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b949e4-c914-4d1e-b85b-e0dc711ede96 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:43.328169 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b949e4-c914-4d1e-b85b-e0dc711ede96 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:43.328260 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b949e4-c914-4d1e-b85b-e0dc711ede96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:43.410040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-93b949e4-c914-4d1e-b85b-e0dc711ede96 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:14:43.411608 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-93b949e4-c914-4d1e-b85b-e0dc711ede96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:43.521513 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 695/1384] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:14:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:14:43.564560 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77e50d-e62b-4dc9-96eb-5581e27454d6 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:14:43.566439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-bf77e50d-e62b-4dc9-96eb-5581e27454d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:43.730318 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77e50d-e62b-4dc9-96eb-5581e27454d6 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:14:43.803539 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:43.863254 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb775d-38b9-439e-b16f-d16cf55ece61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:14:43.864309 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb775d-38b9-439e-b16f-d16cf55ece61 service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:43.864653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb775d-38b9-439e-b16f-d16cf55ece61 service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:43.865268 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb775d-38b9-439e-b16f-d16cf55ece61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:43.867371 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-53eb775d-38b9-439e-b16f-d16cf55ece61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:43.967372 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eb775d-38b9-439e-b16f-d16cf55ece61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:14:43.968925 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eb775d-38b9-439e-b16f-d16cf55ece61 service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:14:43.987686 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 690/1385] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:14:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:14:46.287483 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb933a-9113-4745-b8a9-6a59c8a5303e None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:46.287996 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb933a-9113-4745-b8a9-6a59c8a5303e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:46.288141 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb933a-9113-4745-b8a9-6a59c8a5303e None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:46.290096 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 696/1386] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:14:46.295467 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a83c7-f359-49c9-8efb-1de10d9711be None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:46.295998 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a83c7-f359-49c9-8efb-1de10d9711be None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:46.295998 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a83c7-f359-49c9-8efb-1de10d9711be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:46.364505 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-9c9a83c7-f359-49c9-8efb-1de10d9711be None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:14:46.365705 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9c9a83c7-f359-49c9-8efb-1de10d9711be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:46.400252 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:46.468312 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 691/1387] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:14:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:14:46.512860 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb933a-9113-4745-b8a9-6a59c8a5303e None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:14:46.514440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e1cb933a-9113-4745-b8a9-6a59c8a5303e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:46.592450 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:46.669715 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb933a-9113-4745-b8a9-6a59c8a5303e None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:14:46.796106 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85afcd-208f-4a0f-b5a0-a2199b724398 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:14:46.796799 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85afcd-208f-4a0f-b5a0-a2199b724398 service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:46.797111 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85afcd-208f-4a0f-b5a0-a2199b724398 service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:46.797412 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85afcd-208f-4a0f-b5a0-a2199b724398 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:46.799220 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-de85afcd-208f-4a0f-b5a0-a2199b724398 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:46.842625 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:46.896887 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de85afcd-208f-4a0f-b5a0-a2199b724398 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:14:46.898531 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de85afcd-208f-4a0f-b5a0-a2199b724398 service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:14:46.919030 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 697/1388] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:14:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:14:49.145946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918c492-7abe-4583-b4a1-cf05d60fe25c None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:49.146571 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918c492-7abe-4583-b4a1-cf05d60fe25c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:49.146571 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918c492-7abe-4583-b4a1-cf05d60fe25c None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:49.148069 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 692/1389] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:14:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:14:49.153290 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d1552-5598-4ff2-b5a8-6a4d0d5a4e30 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:49.153741 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d1552-5598-4ff2-b5a8-6a4d0d5a4e30 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:49.153850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d1552-5598-4ff2-b5a8-6a4d0d5a4e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:49.221339 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-1c2d1552-5598-4ff2-b5a8-6a4d0d5a4e30 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:14:49.222592 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-1c2d1552-5598-4ff2-b5a8-6a4d0d5a4e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:49.318916 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 698/1390] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:14:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:14:49.363178 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f918c492-7abe-4583-b4a1-cf05d60fe25c None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:14:49.364685 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f918c492-7abe-4583-b4a1-cf05d60fe25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:49.515119 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f918c492-7abe-4583-b4a1-cf05d60fe25c None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:14:49.532153 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:49.632059 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d7710-89b1-40f5-9996-704769156e3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:14:49.632686 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d7710-89b1-40f5-9996-704769156e3a service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:49.632969 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d7710-89b1-40f5-9996-704769156e3a service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:49.633780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d7710-89b1-40f5-9996-704769156e3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:49.635439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-5c1d7710-89b1-40f5-9996-704769156e3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:49.725362 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1d7710-89b1-40f5-9996-704769156e3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:14:49.728054 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1d7710-89b1-40f5-9996-704769156e3a service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:14:49.746756 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 693/1391] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:14:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:14:49.903333 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a8f18-2fb2-4a4c-81e0-1cc4ecb90409 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:49.903614 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a8f18-2fb2-4a4c-81e0-1cc4ecb90409 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:49.903890 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a8f18-2fb2-4a4c-81e0-1cc4ecb90409 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:49.905620 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 699/1392] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:14:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:14:49.910909 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe44fa9-3f19-4ed3-8273-f38e64dfc18c None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:49.911200 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe44fa9-3f19-4ed3-8273-f38e64dfc18c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:49.911474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe44fa9-3f19-4ed3-8273-f38e64dfc18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:49.939699 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:49.985029 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-3fe44fa9-3f19-4ed3-8273-f38e64dfc18c None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:14:49.986189 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3fe44fa9-3f19-4ed3-8273-f38e64dfc18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:50.083384 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 694/1393] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:14:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:14:50.143580 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e72fd-21a1-4fd0-afd3-05d1b02f8244 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:50.143870 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e72fd-21a1-4fd0-afd3-05d1b02f8244 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:50.144150 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e72fd-21a1-4fd0-afd3-05d1b02f8244 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:50.145503 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 700/1394] 10.222.0.46 () {54 vars in 966 bytes} [Wed Jul 27 04:14:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:14:50.150766 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd6351-1c8e-475f-be6d-16a2f3e97095 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:50.151223 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd6351-1c8e-475f-be6d-16a2f3e97095 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:50.151322 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd6351-1c8e-475f-be6d-16a2f3e97095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:50.229814 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-78bd6351-1c8e-475f-be6d-16a2f3e97095 None None] MFA Rules not processed for user `3e98a41a0f304fc2bb36a18a77a2d729`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:14:50.231091 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-78bd6351-1c8e-475f-be6d-16a2f3e97095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:50.272917 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:50.310321 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 695/1395] 10.222.0.46 () {58 vars in 1063 bytes} [Wed Jul 27 04:14:50 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:14:50.318345 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e72fd-21a1-4fd0-afd3-05d1b02f8244 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:14:50.319844 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-722e72fd-21a1-4fd0-afd3-05d1b02f8244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:50.335091 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:50.443053 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e72fd-21a1-4fd0-afd3-05d1b02f8244 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:14:50.520594 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a20b7c-ba6b-43cf-ae55-2a48667415b2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e98a41a0f304fc2bb36a18a77a2d729', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:14:50.521218 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a20b7c-ba6b-43cf-ae55-2a48667415b2 service glance] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:50.521497 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a20b7c-ba6b-43cf-ae55-2a48667415b2 service glance] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:50.521768 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a20b7c-ba6b-43cf-ae55-2a48667415b2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:50.523480 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-84a20b7c-ba6b-43cf-ae55-2a48667415b2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:50.592892 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:50.635280 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a20b7c-ba6b-43cf-ae55-2a48667415b2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:14:50.636720 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a20b7c-ba6b-43cf-ae55-2a48667415b2 service glance] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:14:50.654852 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 701/1396] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:14:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:14:51.389713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffbe15-c2b2-443f-b71a-9431ad0b0f72 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:51.390235 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffbe15-c2b2-443f-b71a-9431ad0b0f72 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:51.390235 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffbe15-c2b2-443f-b71a-9431ad0b0f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:51.460429 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-57ffbe15-c2b2-443f-b71a-9431ad0b0f72 None None] MFA Rules not processed for user `8ec07ee1efba4e8089b94f24be338f86`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:14:51.462256 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-57ffbe15-c2b2-443f-b71a-9431ad0b0f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:51.549246 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 696/1397] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Jul 27 04:14:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:14:51.593267 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4788f5-8372-4731-a610-20b7ea04cb85 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:51.593764 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4788f5-8372-4731-a610-20b7ea04cb85 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:51.594037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4788f5-8372-4731-a610-20b7ea04cb85 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:51.595471 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 702/1398] 10.222.0.46 () {54 vars in 943 bytes} [Wed Jul 27 04:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:14:51.601830 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343c297-0850-4e3f-9a49-140e3d3c9298 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:51.602165 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343c297-0850-4e3f-9a49-140e3d3c9298 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:51.602450 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343c297-0850-4e3f-9a49-140e3d3c9298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:51.672809 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:51.681362 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-e343c297-0850-4e3f-9a49-140e3d3c9298 None None] MFA Rules not processed for user `57b5fa7517034a39930ce7c9d12e1875`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:14:51.682624 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e343c297-0850-4e3f-9a49-140e3d3c9298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:51.799366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 697/1399] 10.222.0.46 () {58 vars in 1040 bytes} [Wed Jul 27 04:14:51 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:14:51.801920 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1301b67-97df-4d70-9670-ebdb19cbb8b6 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:51.802453 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1301b67-97df-4d70-9670-ebdb19cbb8b6 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:51.802879 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1301b67-97df-4d70-9670-ebdb19cbb8b6 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:51.805351 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 703/1400] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:14:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:14:51.811142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07474e07-77f2-447b-9a63-07c1dcbd4516 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:51.811437 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07474e07-77f2-447b-9a63-07c1dcbd4516 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:51.811746 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07474e07-77f2-447b-9a63-07c1dcbd4516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:51.811876 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1301b67-97df-4d70-9670-ebdb19cbb8b6 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:14:51.813315 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c1301b67-97df-4d70-9670-ebdb19cbb8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:51.881661 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-07474e07-77f2-447b-9a63-07c1dcbd4516 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:14:51.882916 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-07474e07-77f2-447b-9a63-07c1dcbd4516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:51.946937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:51.955740 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1301b67-97df-4d70-9670-ebdb19cbb8b6 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:14:51.984980 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 698/1401] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:14:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:14:51.994861 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:52.029242 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07474e07-77f2-447b-9a63-07c1dcbd4516 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:14:52.030470 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-07474e07-77f2-447b-9a63-07c1dcbd4516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:52.065796 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ca368-05ac-496d-b30d-d034b2ba89e5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57b5fa7517034a39930ce7c9d12e1875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:14:52.066450 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ca368-05ac-496d-b30d-d034b2ba89e5 service swift] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:52.066727 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ca368-05ac-496d-b30d-d034b2ba89e5 service swift] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:52.066999 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ca368-05ac-496d-b30d-d034b2ba89e5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:52.068651 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d74ca368-05ac-496d-b30d-d034b2ba89e5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:52.150482 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07474e07-77f2-447b-9a63-07c1dcbd4516 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:14:52.151264 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74ca368-05ac-496d-b30d-d034b2ba89e5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:14:52.152920 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74ca368-05ac-496d-b30d-d034b2ba89e5 service swift] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:14:52.161173 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 704/1402] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 27 04:14:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 04:14:52.232180 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:52.278354 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6caffe-c4fb-408b-bf34-c1415684328d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:14:52.278990 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6caffe-c4fb-408b-bf34-c1415684328d service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:52.279274 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6caffe-c4fb-408b-bf34-c1415684328d service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:52.279550 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6caffe-c4fb-408b-bf34-c1415684328d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:52.281199 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-cf6caffe-c4fb-408b-bf34-c1415684328d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:52.382481 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6caffe-c4fb-408b-bf34-c1415684328d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:14:52.384238 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6caffe-c4fb-408b-bf34-c1415684328d service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:14:52.409042 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 699/1403] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:14:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 04:14:54.524468 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2812da8-dd80-4bd0-8cd6-fe4c05e9c023 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:54.525277 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2812da8-dd80-4bd0-8cd6-fe4c05e9c023 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:54.525277 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2812da8-dd80-4bd0-8cd6-fe4c05e9c023 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:54.530466 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 705/1404] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:14:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:14:54.531807 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d95f7-b2e9-4dab-b122-accbffd849aa None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:54.532240 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d95f7-b2e9-4dab-b122-accbffd849aa None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:54.532389 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d95f7-b2e9-4dab-b122-accbffd849aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:54.605929 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-fd9d95f7-b2e9-4dab-b122-accbffd849aa None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:14:54.607194 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-fd9d95f7-b2e9-4dab-b122-accbffd849aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:54.647866 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:54.722115 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 700/1405] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:14:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:14:54.765350 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2812da8-dd80-4bd0-8cd6-fe4c05e9c023 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:14:54.766914 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a2812da8-dd80-4bd0-8cd6-fe4c05e9c023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:54.846655 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:54.928113 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2812da8-dd80-4bd0-8cd6-fe4c05e9c023 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:14:55.056163 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54109d7-3797-44d4-ba77-192ebbcb6b93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:14:55.056734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54109d7-3797-44d4-ba77-192ebbcb6b93 service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:55.057014 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54109d7-3797-44d4-ba77-192ebbcb6b93 service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:55.057280 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54109d7-3797-44d4-ba77-192ebbcb6b93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:55.058970 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f54109d7-3797-44d4-ba77-192ebbcb6b93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:55.094219 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:55.158919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54109d7-3797-44d4-ba77-192ebbcb6b93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:14:55.160561 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54109d7-3797-44d4-ba77-192ebbcb6b93 service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:14:55.185044 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 706/1406] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:14:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:14:57.276130 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9b50a-e5fc-477d-825a-d8600fd806c3 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:57.276130 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9b50a-e5fc-477d-825a-d8600fd806c3 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:57.276130 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9b50a-e5fc-477d-825a-d8600fd806c3 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:57.280414 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 701/1407] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:14:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:14:57.282154 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ad9fd-446e-4389-8c17-7382e1002d07 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:57.282599 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ad9fd-446e-4389-8c17-7382e1002d07 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:57.282733 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ad9fd-446e-4389-8c17-7382e1002d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:57.359825 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-de7ad9fd-446e-4389-8c17-7382e1002d07 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:14:57.361060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-de7ad9fd-446e-4389-8c17-7382e1002d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:57.484284 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 707/1408] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:14:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:14:57.532116 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9b50a-e5fc-477d-825a-d8600fd806c3 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:14:57.533881 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c3f9b50a-e5fc-477d-825a-d8600fd806c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:57.838672 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9b50a-e5fc-477d-825a-d8600fd806c3 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:14:57.864679 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:14:57.976405 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578ba0a-17fc-4df2-82a5-f8e5667568bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:14:57.978682 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578ba0a-17fc-4df2-82a5-f8e5667568bf service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:14:57.978966 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578ba0a-17fc-4df2-82a5-f8e5667568bf service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:14:57.979237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578ba0a-17fc-4df2-82a5-f8e5667568bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:14:57.980872 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d578ba0a-17fc-4df2-82a5-f8e5667568bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:14:58.080923 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578ba0a-17fc-4df2-82a5-f8e5667568bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:14:58.083833 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578ba0a-17fc-4df2-82a5-f8e5667568bf service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:14:58.105500 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 702/1409] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:14:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:15:07.477830 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:08.103059 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=88 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:14.448064 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f4ad7-90c4-4147-9a22-f1c4b1874ca3 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:14.448640 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f4ad7-90c4-4147-9a22-f1c4b1874ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:14.448640 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f4ad7-90c4-4147-9a22-f1c4b1874ca3 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:14.450149 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 708/1410] 10.222.0.46 () {54 vars in 918 bytes} [Wed Jul 27 04:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:15:14.455183 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4ebec-1e23-4afa-8594-cab2161ee4f8 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:14.455641 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4ebec-1e23-4afa-8594-cab2161ee4f8 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:14.456214 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4ebec-1e23-4afa-8594-cab2161ee4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:14.530553 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-88d4ebec-1e23-4afa-8594-cab2161ee4f8 None None] MFA Rules not processed for user `8c08fba2d7de42218400c5e652cc3de0`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:15:14.531780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-88d4ebec-1e23-4afa-8594-cab2161ee4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:14.637420 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 703/1411] 10.222.0.46 () {58 vars in 1015 bytes} [Wed Jul 27 04:15:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:15:14.652248 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af4682-a812-49e6-ae5f-816eec0562f3 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:14.652536 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af4682-a812-49e6-ae5f-816eec0562f3 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:14.652818 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af4682-a812-49e6-ae5f-816eec0562f3 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:14.654318 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 709/1412] 10.222.0.46 () {54 vars in 918 bytes} [Wed Jul 27 04:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:15:14.658781 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321494e-f2a2-4b49-b8d3-493d79ce35fc None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:14.659066 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321494e-f2a2-4b49-b8d3-493d79ce35fc None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:14.659339 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321494e-f2a2-4b49-b8d3-493d79ce35fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:14.730234 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:14.733388 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-c321494e-f2a2-4b49-b8d3-493d79ce35fc None None] MFA Rules not processed for user `9dc9ea8f89c14a2a8e3272f750647dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:15:14.735164 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c321494e-f2a2-4b49-b8d3-493d79ce35fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:14.856763 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 704/1413] 10.222.0.46 () {58 vars in 1015 bytes} [Wed Jul 27 04:15:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:15:17.308483 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691064a3-bbc7-43d2-baed-9bb67e5e5122 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:17.309009 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691064a3-bbc7-43d2-baed-9bb67e5e5122 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:17.309009 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691064a3-bbc7-43d2-baed-9bb67e5e5122 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:17.310488 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 04:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:15:17.315665 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778b46a-49a4-40c7-99d0-e38fd9873880 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:17.316065 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778b46a-49a4-40c7-99d0-e38fd9873880 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:17.316216 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778b46a-49a4-40c7-99d0-e38fd9873880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:17.381522 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-3778b46a-49a4-40c7-99d0-e38fd9873880 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:15:17.382914 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3778b46a-49a4-40c7-99d0-e38fd9873880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:17.437328 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:17.480825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 04:15:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:15:17.524284 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691064a3-bbc7-43d2-baed-9bb67e5e5122 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:15:17.525910 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-691064a3-bbc7-43d2-baed-9bb67e5e5122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:17.671750 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691064a3-bbc7-43d2-baed-9bb67e5e5122 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:15:17.742691 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:17.783345 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15aa64-7888-4d07-9d59-4c99f6c7bcf8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:15:17.783961 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15aa64-7888-4d07-9d59-4c99f6c7bcf8 service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:17.784240 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15aa64-7888-4d07-9d59-4c99f6c7bcf8 service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:17.784511 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15aa64-7888-4d07-9d59-4c99f6c7bcf8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:17.786153 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-ab15aa64-7888-4d07-9d59-4c99f6c7bcf8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:17.877032 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab15aa64-7888-4d07-9d59-4c99f6c7bcf8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:15:17.878682 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab15aa64-7888-4d07-9d59-4c99f6c7bcf8 service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:15:17.897356 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 711/1416] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:15:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:15:18.799651 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c71f81-5e0c-493d-878f-04ce6118f016 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:18.800157 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c71f81-5e0c-493d-878f-04ce6118f016 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:18.800157 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c71f81-5e0c-493d-878f-04ce6118f016 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:18.801469 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 04:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:15:18.806679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35208850-2817-4e71-a994-73efb65ccfbb None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:18.807360 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35208850-2817-4e71-a994-73efb65ccfbb None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:18.807607 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35208850-2817-4e71-a994-73efb65ccfbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:18.882751 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-35208850-2817-4e71-a994-73efb65ccfbb None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:15:18.883948 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-35208850-2817-4e71-a994-73efb65ccfbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:18.909875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:18.979949 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 04:15:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:15:19.129754 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c71f81-5e0c-493d-878f-04ce6118f016 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:15:19.131313 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f0c71f81-5e0c-493d-878f-04ce6118f016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:19.279225 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c71f81-5e0c-493d-878f-04ce6118f016 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:15:19.317932 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:19.391227 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de8645-89e8-4423-b8d8-64fa2d6013e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:15:19.391766 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de8645-89e8-4423-b8d8-64fa2d6013e5 service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:19.392045 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de8645-89e8-4423-b8d8-64fa2d6013e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:19.392317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de8645-89e8-4423-b8d8-64fa2d6013e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:19.393958 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-44de8645-89e8-4423-b8d8-64fa2d6013e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:19.483580 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44de8645-89e8-4423-b8d8-64fa2d6013e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:15:19.485163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44de8645-89e8-4423-b8d8-64fa2d6013e5 service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:15:19.503335 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 707/1419] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:15:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:15:20.412815 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f3864-4410-4756-b272-bb04c99a5563 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:20.413383 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f3864-4410-4756-b272-bb04c99a5563 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:20.413383 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f3864-4410-4756-b272-bb04c99a5563 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:20.415231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 04:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:15:20.420256 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888b6b8-ac68-4bef-b57a-125f87e1b7e1 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:20.420659 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888b6b8-ac68-4bef-b57a-125f87e1b7e1 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:20.420863 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888b6b8-ac68-4bef-b57a-125f87e1b7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:20.477073 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:20.499490 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-a888b6b8-ac68-4bef-b57a-125f87e1b7e1 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:15:20.500684 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-a888b6b8-ac68-4bef-b57a-125f87e1b7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:20.596318 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 04:15:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:15:20.752048 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f3864-4410-4756-b272-bb04c99a5563 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:15:20.753465 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-218f3864-4410-4756-b272-bb04c99a5563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:20.896424 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f3864-4410-4756-b272-bb04c99a5563 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:15:20.908137 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:21.006553 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f86b05-21e0-45ff-9d94-89705e54b753 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:15:21.007264 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f86b05-21e0-45ff-9d94-89705e54b753 service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:21.007550 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f86b05-21e0-45ff-9d94-89705e54b753 service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:21.007854 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f86b05-21e0-45ff-9d94-89705e54b753 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:21.009446 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b2f86b05-21e0-45ff-9d94-89705e54b753 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:21.096003 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f86b05-21e0-45ff-9d94-89705e54b753 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:15:21.097573 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f86b05-21e0-45ff-9d94-89705e54b753 service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:15:21.116114 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 714/1422] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:15:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:15:21.978957 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4a977-99f2-4843-b0ab-a889c27fa2db None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:21.979494 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4a977-99f2-4843-b0ab-a889c27fa2db None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:21.979494 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4a977-99f2-4843-b0ab-a889c27fa2db None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:21.980874 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 04:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:15:21.985873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89c8e1-a590-4088-806c-6b3efcf00f58 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:21.986285 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89c8e1-a590-4088-806c-6b3efcf00f58 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:21.986467 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89c8e1-a590-4088-806c-6b3efcf00f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:22.006144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:22.063335 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-7e89c8e1-a590-4088-806c-6b3efcf00f58 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:15:22.064533 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7e89c8e1-a590-4088-806c-6b3efcf00f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:22.157710 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 04:15:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:15:22.310165 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef4a977-99f2-4843-b0ab-a889c27fa2db None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:15:22.311612 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-5ef4a977-99f2-4843-b0ab-a889c27fa2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:22.421099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:22.457789 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef4a977-99f2-4843-b0ab-a889c27fa2db None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:15:22.568692 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e687923-dfaa-4338-8f34-ddfe8807932e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:15:22.569234 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e687923-dfaa-4338-8f34-ddfe8807932e service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:22.569511 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e687923-dfaa-4338-8f34-ddfe8807932e service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:22.569778 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e687923-dfaa-4338-8f34-ddfe8807932e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:22.572210 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-4e687923-dfaa-4338-8f34-ddfe8807932e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:22.646365 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:22.660246 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e687923-dfaa-4338-8f34-ddfe8807932e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:15:22.661833 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e687923-dfaa-4338-8f34-ddfe8807932e service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:15:22.680516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 710/1425] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:15:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:15:23.532948 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f6d92-9154-4cc9-908c-c35ceb3b0bc5 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:23.533539 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f6d92-9154-4cc9-908c-c35ceb3b0bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:23.533539 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f6d92-9154-4cc9-908c-c35ceb3b0bc5 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:23.534907 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 04:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:15:23.540615 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41453a-50f2-4f29-aeae-ea4d96cf1748 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:23.541049 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41453a-50f2-4f29-aeae-ea4d96cf1748 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:23.541195 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41453a-50f2-4f29-aeae-ea4d96cf1748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:23.621207 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-ec41453a-50f2-4f29-aeae-ea4d96cf1748 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:15:23.622453 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ec41453a-50f2-4f29-aeae-ea4d96cf1748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:23.717740 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 04:15:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:15:23.869154 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f6d92-9154-4cc9-908c-c35ceb3b0bc5 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:15:23.870761 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e91f6d92-9154-4cc9-908c-c35ceb3b0bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:23.954718 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:24.016855 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f6d92-9154-4cc9-908c-c35ceb3b0bc5 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:15:24.127570 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f05de-9214-441a-b274-257770c3a6e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:15:24.128228 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f05de-9214-441a-b274-257770c3a6e7 service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:24.128509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f05de-9214-441a-b274-257770c3a6e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:24.128784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f05de-9214-441a-b274-257770c3a6e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:24.130439 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-036f05de-9214-441a-b274-257770c3a6e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:24.178191 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:24.219232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f05de-9214-441a-b274-257770c3a6e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:15:24.220834 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f05de-9214-441a-b274-257770c3a6e7 service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:15:24.239065 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 717/1428] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:15:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:15:25.046910 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73838f4-c7d2-41a6-b47a-22eac91b7e27 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:25.047454 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73838f4-c7d2-41a6-b47a-22eac91b7e27 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:25.047588 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73838f4-c7d2-41a6-b47a-22eac91b7e27 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:25.048834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 04:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:15:25.053689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7716b-4f3b-42ec-bd9d-eb32fc2257dc None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:25.054147 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7716b-4f3b-42ec-bd9d-eb32fc2257dc None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:25.054297 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7716b-4f3b-42ec-bd9d-eb32fc2257dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:25.131346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-98d7716b-4f3b-42ec-bd9d-eb32fc2257dc None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:15:25.132573 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-98d7716b-4f3b-42ec-bd9d-eb32fc2257dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:25.232231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 04:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:15:25.383827 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73838f4-c7d2-41a6-b47a-22eac91b7e27 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:15:25.385335 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-a73838f4-c7d2-41a6-b47a-22eac91b7e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:25.433631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:25.533750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73838f4-c7d2-41a6-b47a-22eac91b7e27 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:15:25.646640 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a0c45-db95-4f48-be7f-537a099fda7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:15:25.647181 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a0c45-db95-4f48-be7f-537a099fda7b service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:25.647464 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a0c45-db95-4f48-be7f-537a099fda7b service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:25.647736 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a0c45-db95-4f48-be7f-537a099fda7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:25.650847 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e73a0c45-db95-4f48-be7f-537a099fda7b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:25.657228 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:25.739919 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a0c45-db95-4f48-be7f-537a099fda7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:15:25.741548 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a0c45-db95-4f48-be7f-537a099fda7b service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:15:25.759819 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 713/1431] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:15:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:15:26.580388 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecc51f-f7a6-42dc-9031-52844785ffa5 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:26.580938 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecc51f-f7a6-42dc-9031-52844785ffa5 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:26.580938 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecc51f-f7a6-42dc-9031-52844785ffa5 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:26.582347 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 04:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:15:26.587958 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615740c-4dc1-4ae0-a12d-a9bccf65e6c4 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:26.588394 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615740c-4dc1-4ae0-a12d-a9bccf65e6c4 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:26.588526 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615740c-4dc1-4ae0-a12d-a9bccf65e6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:26.668035 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-b615740c-4dc1-4ae0-a12d-a9bccf65e6c4 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:15:26.669246 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b615740c-4dc1-4ae0-a12d-a9bccf65e6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:26.735871 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:26.765626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 04:15:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:15:26.923858 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ecc51f-f7a6-42dc-9031-52844785ffa5 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:15:26.925362 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d6ecc51f-f7a6-42dc-9031-52844785ffa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:26.943662 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:27.072700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ecc51f-f7a6-42dc-9031-52844785ffa5 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:15:27.160193 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:27.186156 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47d3a68-c1f8-45bf-8167-d1e4f99592e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:15:27.186782 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d3a68-c1f8-45bf-8167-d1e4f99592e1 service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:27.187061 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d3a68-c1f8-45bf-8167-d1e4f99592e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:27.187331 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d3a68-c1f8-45bf-8167-d1e4f99592e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:27.188929 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f47d3a68-c1f8-45bf-8167-d1e4f99592e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:27.277686 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47d3a68-c1f8-45bf-8167-d1e4f99592e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:15:27.279348 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47d3a68-c1f8-45bf-8167-d1e4f99592e1 service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:15:27.297409 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 720/1434] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:15:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:15:28.104939 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff64f3-be1d-4561-9e68-5dce3eb2d7ad None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:28.105561 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff64f3-be1d-4561-9e68-5dce3eb2d7ad None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:28.105561 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff64f3-be1d-4561-9e68-5dce3eb2d7ad None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:28.106900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 04:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:15:28.111766 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f6d7c-b347-405b-9557-41d5a40af689 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:28.112203 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f6d7c-b347-405b-9557-41d5a40af689 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:28.112309 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f6d7c-b347-405b-9557-41d5a40af689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:28.189704 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-d87f6d7c-b347-405b-9557-41d5a40af689 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:15:28.191196 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d87f6d7c-b347-405b-9557-41d5a40af689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:28.232923 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:28.286817 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 04:15:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:15:28.437020 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff64f3-be1d-4561-9e68-5dce3eb2d7ad None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:15:28.438604 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-66ff64f3-be1d-4561-9e68-5dce3eb2d7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:28.598199 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff64f3-be1d-4561-9e68-5dce3eb2d7ad None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:15:28.652157 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:28.711582 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b952df-d9f8-4a33-8418-c03d632e1b3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:15:28.712205 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b952df-d9f8-4a33-8418-c03d632e1b3c service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:28.712490 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b952df-d9f8-4a33-8418-c03d632e1b3c service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:28.712768 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b952df-d9f8-4a33-8418-c03d632e1b3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:28.714419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-36b952df-d9f8-4a33-8418-c03d632e1b3c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:28.804207 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b952df-d9f8-4a33-8418-c03d632e1b3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:15:28.806680 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b952df-d9f8-4a33-8418-c03d632e1b3c service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:15:28.825065 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 716/1437] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:15:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:15:29.636681 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9307c3-b1e7-4077-b7df-cfee42f017fa None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:29.637229 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9307c3-b1e7-4077-b7df-cfee42f017fa None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:29.637229 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9307c3-b1e7-4077-b7df-cfee42f017fa None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:29.638637 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 04:15:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:15:29.643586 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7978b0-d6d6-4008-8eba-b75401768da0 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:29.644025 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7978b0-d6d6-4008-8eba-b75401768da0 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:29.644128 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7978b0-d6d6-4008-8eba-b75401768da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:29.719633 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-7e7978b0-d6d6-4008-8eba-b75401768da0 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:15:29.720771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-7e7978b0-d6d6-4008-8eba-b75401768da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:29.722798 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:29.819549 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 04:15:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:15:29.973464 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9307c3-b1e7-4077-b7df-cfee42f017fa None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:15:29.975045 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-ed9307c3-b1e7-4077-b7df-cfee42f017fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:30.118073 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9307c3-b1e7-4077-b7df-cfee42f017fa None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:15:30.144656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:30.231651 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54365d-6da4-4d6d-87d8-fe7073692cd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:15:30.232197 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54365d-6da4-4d6d-87d8-fe7073692cd9 service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:30.232478 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54365d-6da4-4d6d-87d8-fe7073692cd9 service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:30.232747 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54365d-6da4-4d6d-87d8-fe7073692cd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:30.234412 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-fa54365d-6da4-4d6d-87d8-fe7073692cd9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:30.322504 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa54365d-6da4-4d6d-87d8-fe7073692cd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:15:30.324090 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa54365d-6da4-4d6d-87d8-fe7073692cd9 service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:15:30.341712 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 723/1440] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:15:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:15:32.238038 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c415f3-6235-4d2e-8dc7-b88932950b93 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:32.238613 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c415f3-6235-4d2e-8dc7-b88932950b93 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:32.238613 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c415f3-6235-4d2e-8dc7-b88932950b93 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:32.239933 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 04:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:15:32.245728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073f15e-886d-4020-8b25-37a87e8a646a None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:32.246179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073f15e-886d-4020-8b25-37a87e8a646a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:32.246307 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073f15e-886d-4020-8b25-37a87e8a646a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:32.286147 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:32.311462 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-e073f15e-886d-4020-8b25-37a87e8a646a None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:15:32.312637 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e073f15e-886d-4020-8b25-37a87e8a646a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:32.527169 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 04:15:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:15:32.566367 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c415f3-6235-4d2e-8dc7-b88932950b93 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:15:32.567873 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-08c415f3-6235-4d2e-8dc7-b88932950b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:32.669331 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c415f3-6235-4d2e-8dc7-b88932950b93 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:15:32.700605 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:32.740966 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbdbc76-85ce-425a-ba22-f8399e1caef4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:15:32.741576 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdbc76-85ce-425a-ba22-f8399e1caef4 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:32.741920 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdbc76-85ce-425a-ba22-f8399e1caef4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:32.742223 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdbc76-85ce-425a-ba22-f8399e1caef4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:32.743820 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-dbbdbc76-85ce-425a-ba22-f8399e1caef4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:32.851360 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbdbc76-85ce-425a-ba22-f8399e1caef4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:15:32.852809 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbdbc76-85ce-425a-ba22-f8399e1caef4 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:15:32.870912 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 719/1443] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:15:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:15:35.022356 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df384047-3bfb-47cc-bf25-ac44cf5f934b None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:35.022919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df384047-3bfb-47cc-bf25-ac44cf5f934b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:35.022919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df384047-3bfb-47cc-bf25-ac44cf5f934b None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:35.024283 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 04:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:15:35.029577 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4cff2c-164c-48cb-a09f-e8bd7db6993e None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:35.030026 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4cff2c-164c-48cb-a09f-e8bd7db6993e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:35.030164 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4cff2c-164c-48cb-a09f-e8bd7db6993e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:35.096211 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-fa4cff2c-164c-48cb-a09f-e8bd7db6993e None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:15:35.097438 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-fa4cff2c-164c-48cb-a09f-e8bd7db6993e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:35.110689 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:35.194531 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 04:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:15:35.233078 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df384047-3bfb-47cc-bf25-ac44cf5f934b None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:15:35.234680 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-df384047-3bfb-47cc-bf25-ac44cf5f934b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:35.336414 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df384047-3bfb-47cc-bf25-ac44cf5f934b None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:15:35.348165 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:35.408285 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409766b6-87c9-4ef8-99ca-a6fa4789486d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:15:35.408899 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409766b6-87c9-4ef8-99ca-a6fa4789486d service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:35.409175 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409766b6-87c9-4ef8-99ca-a6fa4789486d service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:35.409442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409766b6-87c9-4ef8-99ca-a6fa4789486d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:35.411058 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-409766b6-87c9-4ef8-99ca-a6fa4789486d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:35.517325 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409766b6-87c9-4ef8-99ca-a6fa4789486d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:15:35.518784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409766b6-87c9-4ef8-99ca-a6fa4789486d service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:15:35.536099 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 726/1446] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:15:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:15:37.827535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e05999-a989-41da-a317-083e92a48fb8 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:37.828040 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e05999-a989-41da-a317-083e92a48fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:37.828040 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e05999-a989-41da-a317-083e92a48fb8 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:37.829353 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 04:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:15:37.834448 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4fae6-9964-4967-84d1-a2459bb4de8e None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:37.834876 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4fae6-9964-4967-84d1-a2459bb4de8e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:37.835295 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4fae6-9964-4967-84d1-a2459bb4de8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:37.882135 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:37.901347 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-e7b4fae6-9964-4967-84d1-a2459bb4de8e None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:15:37.902590 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e7b4fae6-9964-4967-84d1-a2459bb4de8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:38.002377 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 04:15:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:15:38.041730 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e05999-a989-41da-a317-083e92a48fb8 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:15:38.043280 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-20e05999-a989-41da-a317-083e92a48fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:38.144840 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e05999-a989-41da-a317-083e92a48fb8 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:15:38.185458 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:38.219104 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48362b7b-700c-49c0-8d10-ef3113251299 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:15:38.219809 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48362b7b-700c-49c0-8d10-ef3113251299 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:38.220109 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48362b7b-700c-49c0-8d10-ef3113251299 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:38.220451 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48362b7b-700c-49c0-8d10-ef3113251299 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:38.222225 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-48362b7b-700c-49c0-8d10-ef3113251299 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:38.328023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48362b7b-700c-49c0-8d10-ef3113251299 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:15:38.329452 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48362b7b-700c-49c0-8d10-ef3113251299 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:15:38.347420 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 722/1449] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:15:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:15:40.448177 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674f99a-6e98-41cc-812e-01772b3d28d2 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:40.448744 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674f99a-6e98-41cc-812e-01772b3d28d2 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:40.448853 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674f99a-6e98-41cc-812e-01772b3d28d2 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:40.450186 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 04:15:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:15:40.455560 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd64d4f-f05d-4b2e-ab1b-92539a68e5ac None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:40.456003 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd64d4f-f05d-4b2e-ab1b-92539a68e5ac None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:40.456595 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd64d4f-f05d-4b2e-ab1b-92539a68e5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:40.521820 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-5cd64d4f-f05d-4b2e-ab1b-92539a68e5ac None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:15:40.523074 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-5cd64d4f-f05d-4b2e-ab1b-92539a68e5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:40.543302 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:40.620890 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 04:15:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:15:40.659078 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7674f99a-6e98-41cc-812e-01772b3d28d2 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:15:40.660623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7674f99a-6e98-41cc-812e-01772b3d28d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:40.762225 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7674f99a-6e98-41cc-812e-01772b3d28d2 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:15:40.781495 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:40.834581 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa38c0-0f10-471b-9613-b0fd61c2dd98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:15:40.835205 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa38c0-0f10-471b-9613-b0fd61c2dd98 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:40.835572 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa38c0-0f10-471b-9613-b0fd61c2dd98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:40.835877 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa38c0-0f10-471b-9613-b0fd61c2dd98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:40.837601 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c9fa38c0-0f10-471b-9613-b0fd61c2dd98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:40.943486 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fa38c0-0f10-471b-9613-b0fd61c2dd98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:15:40.945461 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fa38c0-0f10-471b-9613-b0fd61c2dd98 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:15:40.964001 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 729/1452] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:15:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:15:43.084984 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942bb470-d975-4472-ac12-5c4497826b14 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:43.085556 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942bb470-d975-4472-ac12-5c4497826b14 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:43.085656 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942bb470-d975-4472-ac12-5c4497826b14 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:43.087066 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 04:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:15:43.092336 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea40647-ee95-4f99-84ea-f530bc136018 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:43.092742 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea40647-ee95-4f99-84ea-f530bc136018 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:43.092901 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea40647-ee95-4f99-84ea-f530bc136018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:43.155327 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:43.158410 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-7ea40647-ee95-4f99-84ea-f530bc136018 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:15:43.159929 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7ea40647-ee95-4f99-84ea-f530bc136018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:43.257770 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 04:15:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:15:43.302111 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942bb470-d975-4472-ac12-5c4497826b14 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:15:43.303622 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-942bb470-d975-4472-ac12-5c4497826b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:43.407808 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942bb470-d975-4472-ac12-5c4497826b14 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:15:43.456332 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:43.482418 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb7c63-10c9-498f-9d1d-b55cb850591c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:15:43.483047 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb7c63-10c9-498f-9d1d-b55cb850591c service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:43.483329 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb7c63-10c9-498f-9d1d-b55cb850591c service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:43.483606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb7c63-10c9-498f-9d1d-b55cb850591c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:43.485233 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-95fb7c63-10c9-498f-9d1d-b55cb850591c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:43.593726 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fb7c63-10c9-498f-9d1d-b55cb850591c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:15:43.595212 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fb7c63-10c9-498f-9d1d-b55cb850591c service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:15:43.614684 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 725/1455] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:15:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:15:45.748010 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0534f57-01a2-48ef-b138-3de3a3cbeb59 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:45.748521 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0534f57-01a2-48ef-b138-3de3a3cbeb59 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:45.748521 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0534f57-01a2-48ef-b138-3de3a3cbeb59 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:45.749822 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 04:15:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:15:45.755075 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1797234-cc16-4f0c-8718-f8cd984640e8 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:45.755467 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1797234-cc16-4f0c-8718-f8cd984640e8 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:45.755630 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1797234-cc16-4f0c-8718-f8cd984640e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:45.822886 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-b1797234-cc16-4f0c-8718-f8cd984640e8 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:15:45.824113 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b1797234-cc16-4f0c-8718-f8cd984640e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:45.853991 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:45.921469 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 04:15:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:15:45.959728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0534f57-01a2-48ef-b138-3de3a3cbeb59 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:15:45.962191 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a0534f57-01a2-48ef-b138-3de3a3cbeb59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:46.074545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0534f57-01a2-48ef-b138-3de3a3cbeb59 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:15:46.110553 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:46.161046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0883b4-4be6-469f-b5f0-f434ae366362 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:15:46.161676 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0883b4-4be6-469f-b5f0-f434ae366362 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:46.161956 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0883b4-4be6-469f-b5f0-f434ae366362 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:46.162256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0883b4-4be6-469f-b5f0-f434ae366362 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:46.164183 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-ef0883b4-4be6-469f-b5f0-f434ae366362 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:46.292190 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0883b4-4be6-469f-b5f0-f434ae366362 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:15:46.293635 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0883b4-4be6-469f-b5f0-f434ae366362 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:15:46.311882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 732/1458] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:15:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:15:48.506393 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f10212-09b1-41b9-bcc6-16e8d6a10fbc None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:48.506896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f10212-09b1-41b9-bcc6-16e8d6a10fbc None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:48.507019 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f10212-09b1-41b9-bcc6-16e8d6a10fbc None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:48.508825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 04:15:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:15:48.514057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d0697-e0d8-4289-b646-beb45f5c3ff0 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:48.514446 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d0697-e0d8-4289-b646-beb45f5c3ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:48.514611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d0697-e0d8-4289-b646-beb45f5c3ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:48.580945 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-c24d0697-e0d8-4289-b646-beb45f5c3ff0 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:15:48.582234 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c24d0697-e0d8-4289-b646-beb45f5c3ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:48.591049 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:48.678465 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 04:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:15:48.716840 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f10212-09b1-41b9-bcc6-16e8d6a10fbc None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:15:48.718437 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-90f10212-09b1-41b9-bcc6-16e8d6a10fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:48.822140 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f10212-09b1-41b9-bcc6-16e8d6a10fbc None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:15:48.876568 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:48.895541 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244ca41-809c-47c4-b458-3de691e6ec9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:15:48.896155 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244ca41-809c-47c4-b458-3de691e6ec9b service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:48.896431 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244ca41-809c-47c4-b458-3de691e6ec9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:48.896701 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244ca41-809c-47c4-b458-3de691e6ec9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:48.898509 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-4244ca41-809c-47c4-b458-3de691e6ec9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:49.004281 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4244ca41-809c-47c4-b458-3de691e6ec9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:15:49.005630 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4244ca41-809c-47c4-b458-3de691e6ec9b service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:15:49.023389 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 728/1461] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:15:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:15:51.074270 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff7274-3e59-4e28-b2cd-e37244dc9ae9 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:51.074836 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff7274-3e59-4e28-b2cd-e37244dc9ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:51.074836 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff7274-3e59-4e28-b2cd-e37244dc9ae9 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:51.076200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 04:15:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:15:51.081618 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e95ea5-1c03-4cba-9f3e-089add4f8dfa None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:51.082065 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e95ea5-1c03-4cba-9f3e-089add4f8dfa None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:51.082185 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e95ea5-1c03-4cba-9f3e-089add4f8dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:51.150647 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-18e95ea5-1c03-4cba-9f3e-089add4f8dfa None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:15:51.151856 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-18e95ea5-1c03-4cba-9f3e-089add4f8dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:51.190075 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:51.249345 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 04:15:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:15:51.287106 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff7274-3e59-4e28-b2cd-e37244dc9ae9 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:15:51.288676 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-14ff7274-3e59-4e28-b2cd-e37244dc9ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:51.389876 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff7274-3e59-4e28-b2cd-e37244dc9ae9 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:15:51.426413 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:51.464330 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d90dfb-da25-4739-abe3-f9db5b18b249 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:15:51.464961 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d90dfb-da25-4739-abe3-f9db5b18b249 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:51.465244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d90dfb-da25-4739-abe3-f9db5b18b249 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:51.465526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d90dfb-da25-4739-abe3-f9db5b18b249 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:51.467203 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a1d90dfb-da25-4739-abe3-f9db5b18b249 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:51.576383 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d90dfb-da25-4739-abe3-f9db5b18b249 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:15:51.577809 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d90dfb-da25-4739-abe3-f9db5b18b249 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:15:51.595801 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 735/1464] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:15:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:15:53.712058 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982df6d-31bd-4586-80d7-2202b62d4dd4 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:53.712525 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982df6d-31bd-4586-80d7-2202b62d4dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:53.712525 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982df6d-31bd-4586-80d7-2202b62d4dd4 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:53.713876 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 04:15:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:15:53.719266 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce062e0-bde0-4280-b4ca-52b651606eb4 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:53.719637 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce062e0-bde0-4280-b4ca-52b651606eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:53.719819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce062e0-bde0-4280-b4ca-52b651606eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:53.782969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-2ce062e0-bde0-4280-b4ca-52b651606eb4 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:15:53.784337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2ce062e0-bde0-4280-b4ca-52b651606eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:53.802341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:53.879282 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 04:15:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:15:53.916258 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0982df6d-31bd-4586-80d7-2202b62d4dd4 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:15:53.917798 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0982df6d-31bd-4586-80d7-2202b62d4dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:54.021945 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0982df6d-31bd-4586-80d7-2202b62d4dd4 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:15:54.091403 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:54.098592 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a072ce-bd0d-48c4-b90e-94b94ca4045e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:15:54.099212 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a072ce-bd0d-48c4-b90e-94b94ca4045e service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:54.099509 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a072ce-bd0d-48c4-b90e-94b94ca4045e service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:54.099780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a072ce-bd0d-48c4-b90e-94b94ca4045e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:54.102039 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-19a072ce-bd0d-48c4-b90e-94b94ca4045e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:54.210682 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a072ce-bd0d-48c4-b90e-94b94ca4045e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:15:54.212373 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a072ce-bd0d-48c4-b90e-94b94ca4045e service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:15:54.230672 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 731/1467] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:15:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:15:56.227253 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f366d98a-46dd-40aa-80fa-9fed04221941 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:56.227919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f366d98a-46dd-40aa-80fa-9fed04221941 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:56.227919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f366d98a-46dd-40aa-80fa-9fed04221941 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:56.229137 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 04:15:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:15:56.234323 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da79d0-0f82-4a8c-9dc8-49ddada8a36e None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:56.234782 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da79d0-0f82-4a8c-9dc8-49ddada8a36e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:56.234887 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da79d0-0f82-4a8c-9dc8-49ddada8a36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:56.299453 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-19da79d0-0f82-4a8c-9dc8-49ddada8a36e None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:15:56.300652 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-19da79d0-0f82-4a8c-9dc8-49ddada8a36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:56.348746 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:56.397352 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 04:15:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:15:56.435486 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f366d98a-46dd-40aa-80fa-9fed04221941 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:15:56.437070 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f366d98a-46dd-40aa-80fa-9fed04221941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:56.540642 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f366d98a-46dd-40aa-80fa-9fed04221941 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:15:56.586738 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:15:56.611539 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431a29d-9fde-462c-81eb-ad36697f7e47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:15:56.612156 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431a29d-9fde-462c-81eb-ad36697f7e47 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:15:56.612433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431a29d-9fde-462c-81eb-ad36697f7e47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:15:56.612702 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431a29d-9fde-462c-81eb-ad36697f7e47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:15:56.614333 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8431a29d-9fde-462c-81eb-ad36697f7e47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:15:56.719380 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8431a29d-9fde-462c-81eb-ad36697f7e47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:15:56.720795 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8431a29d-9fde-462c-81eb-ad36697f7e47 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:15:56.745160 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 738/1470] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:15:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:16:06.395530 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=22 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:16:06.742877 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=66 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:16:16.378898 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52d637-7def-46db-96d8-e1fd5d8f5879 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:16:16.379438 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52d637-7def-46db-96d8-e1fd5d8f5879 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:16:16.379438 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52d637-7def-46db-96d8-e1fd5d8f5879 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:16:16.380698 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 733/1471] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:16:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:16:16.385799 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d6d08-e1fa-4afd-865c-f180bdb8ab4e None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:16:16.386229 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d6d08-e1fa-4afd-865c-f180bdb8ab4e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:16:16.386411 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d6d08-e1fa-4afd-865c-f180bdb8ab4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:16:16.451747 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-5c6d6d08-e1fa-4afd-865c-f180bdb8ab4e None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:16:16.453005 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5c6d6d08-e1fa-4afd-865c-f180bdb8ab4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:16:16.548214 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 739/1472] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:16:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:16:16.630124 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d39f6-2f72-4f3c-ad45-ba21c87265e2 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:16:16.630347 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d39f6-2f72-4f3c-ad45-ba21c87265e2 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:16:16.630619 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d39f6-2f72-4f3c-ad45-ba21c87265e2 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:16:16.631942 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 734/1473] 10.222.0.46 () {54 vars in 966 bytes} [Wed Jul 27 04:16:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:16:16.637421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a05dae-f1e2-4838-97e5-f02e3611b887 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:16:16.637710 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a05dae-f1e2-4838-97e5-f02e3611b887 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:16:16.638011 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a05dae-f1e2-4838-97e5-f02e3611b887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:16:16.711902 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-54a05dae-f1e2-4838-97e5-f02e3611b887 None None] MFA Rules not processed for user `3e98a41a0f304fc2bb36a18a77a2d729`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:16:16.713105 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-54a05dae-f1e2-4838-97e5-f02e3611b887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:16:16.733692 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:16:16.788730 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 740/1474] 10.222.0.46 () {58 vars in 1063 bytes} [Wed Jul 27 04:16:16 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:16:16.796383 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d39f6-2f72-4f3c-ad45-ba21c87265e2 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:16:16.797853 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-896d39f6-2f72-4f3c-ad45-ba21c87265e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:16:16.900420 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d39f6-2f72-4f3c-ad45-ba21c87265e2 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:16:16.973672 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa9c36-18c9-4a17-a2d4-a9635d9c210c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e98a41a0f304fc2bb36a18a77a2d729', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:16:16.974315 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa9c36-18c9-4a17-a2d4-a9635d9c210c service glance] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:16:16.974598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa9c36-18c9-4a17-a2d4-a9635d9c210c service glance] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:16:16.974869 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa9c36-18c9-4a17-a2d4-a9635d9c210c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:16:16.977628 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-deaa9c36-18c9-4a17-a2d4-a9635d9c210c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:16:17.031365 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:16:17.088142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa9c36-18c9-4a17-a2d4-a9635d9c210c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:16:17.089587 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa9c36-18c9-4a17-a2d4-a9635d9c210c service glance] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:16:17.108542 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 735/1475] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:16:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:16:19.205271 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61559754-e91a-448c-806e-ffff8d24d1e3 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:16:19.205802 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61559754-e91a-448c-806e-ffff8d24d1e3 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:16:19.205802 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61559754-e91a-448c-806e-ffff8d24d1e3 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:16:19.207204 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 741/1476] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:16:19.213214 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b6162-e48b-4222-aaad-2694ffba47a0 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:16:19.213639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b6162-e48b-4222-aaad-2694ffba47a0 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:16:19.213769 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b6162-e48b-4222-aaad-2694ffba47a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:16:19.278529 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-651b6162-e48b-4222-aaad-2694ffba47a0 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:16:19.279725 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-651b6162-e48b-4222-aaad-2694ffba47a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:16:19.375831 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 736/1477] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:16:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:16:19.399326 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61559754-e91a-448c-806e-ffff8d24d1e3 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:16:19.400859 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-61559754-e91a-448c-806e-ffff8d24d1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:16:19.526873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61559754-e91a-448c-806e-ffff8d24d1e3 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:16:19.575803 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:16:19.620779 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa221a-b609-494e-afed-c19caf76f696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:16:19.621397 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa221a-b609-494e-afed-c19caf76f696 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:16:19.621678 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa221a-b609-494e-afed-c19caf76f696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:16:19.621949 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa221a-b609-494e-afed-c19caf76f696 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:16:19.622681 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fa221a-b609-494e-afed-c19caf76f696 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:16:19.624406 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: WARNING py.warnings [None req-73fa221a-b609-494e-afed-c19caf76f696 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 04:16:19.624406 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: warnings.warn(msg) Jul 27 04:16:19.624406 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: Jul 27 04:16:19.625536 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fa221a-b609-494e-afed-c19caf76f696 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:16:19.644704 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 742/1478] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 27 04:16:19 2022] GET /identity/v3/projects => generated 2553 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:16:21.704081 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd54df18-b9e9-431c-bb64-d0118b132083 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:16:21.704569 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd54df18-b9e9-431c-bb64-d0118b132083 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:16:21.704676 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd54df18-b9e9-431c-bb64-d0118b132083 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:16:21.706098 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 737/1479] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:16:21.711912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae19de-9486-415d-b654-1e9f86f1992b None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:16:21.712351 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae19de-9486-415d-b654-1e9f86f1992b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:16:21.712465 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae19de-9486-415d-b654-1e9f86f1992b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:16:21.775777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-01ae19de-9486-415d-b654-1e9f86f1992b None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:16:21.776990 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-01ae19de-9486-415d-b654-1e9f86f1992b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:16:21.873251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 743/1480] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:16:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:16:21.916628 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd54df18-b9e9-431c-bb64-d0118b132083 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:16:21.918237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-bd54df18-b9e9-431c-bb64-d0118b132083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:16:21.927849 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:16:22.061874 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd54df18-b9e9-431c-bb64-d0118b132083 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:16:22.142979 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:16:22.176955 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2a528-6d5d-4d1c-976b-15a5a4fbca32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:16:22.177574 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2a528-6d5d-4d1c-976b-15a5a4fbca32 service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:16:22.177852 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2a528-6d5d-4d1c-976b-15a5a4fbca32 service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:16:22.178165 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2a528-6d5d-4d1c-976b-15a5a4fbca32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:16:22.181023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-43c2a528-6d5d-4d1c-976b-15a5a4fbca32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:16:22.270293 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c2a528-6d5d-4d1c-976b-15a5a4fbca32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:16:22.271878 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c2a528-6d5d-4d1c-976b-15a5a4fbca32 service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:16:22.290204 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 738/1481] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:16:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:16:24.485138 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097db9cc-8a41-45dd-8f9d-7ee77459643a None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:16:24.485656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097db9cc-8a41-45dd-8f9d-7ee77459643a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:16:24.485764 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097db9cc-8a41-45dd-8f9d-7ee77459643a None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:16:24.487100 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 744/1482] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:16:24.492355 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c32d9-b76c-460c-a1cb-f2a0fa7b1136 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:16:24.492760 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c32d9-b76c-460c-a1cb-f2a0fa7b1136 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:16:24.492918 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c32d9-b76c-460c-a1cb-f2a0fa7b1136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:16:24.558882 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-2c3c32d9-b76c-460c-a1cb-f2a0fa7b1136 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:16:24.560102 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2c3c32d9-b76c-460c-a1cb-f2a0fa7b1136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:16:24.606623 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:16:24.658640 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 739/1483] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:16:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:16:24.695654 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097db9cc-8a41-45dd-8f9d-7ee77459643a None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:16:24.697158 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-097db9cc-8a41-45dd-8f9d-7ee77459643a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:16:24.720053 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:16:24.804791 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097db9cc-8a41-45dd-8f9d-7ee77459643a None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:16:24.875599 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec313ab2-edd6-4d34-8944-0d2b67a55231 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:16:24.876212 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec313ab2-edd6-4d34-8944-0d2b67a55231 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:16:24.876489 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec313ab2-edd6-4d34-8944-0d2b67a55231 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:16:24.876759 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec313ab2-edd6-4d34-8944-0d2b67a55231 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:16:24.878395 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-ec313ab2-edd6-4d34-8944-0d2b67a55231 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:16:24.951853 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:16:24.987970 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec313ab2-edd6-4d34-8944-0d2b67a55231 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:16:24.989321 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec313ab2-edd6-4d34-8944-0d2b67a55231 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:16:25.007395 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 745/1484] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:16:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:16:27.045997 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4cd998-fa7a-4515-8491-c0b7c8c39e58 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:16:27.046642 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4cd998-fa7a-4515-8491-c0b7c8c39e58 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:16:27.046642 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4cd998-fa7a-4515-8491-c0b7c8c39e58 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:16:27.047917 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 740/1485] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:16:27.053246 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74f6ab-7feb-44ee-8475-79b96995badf None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:16:27.054245 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74f6ab-7feb-44ee-8475-79b96995badf None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:16:27.054353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74f6ab-7feb-44ee-8475-79b96995badf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:16:27.118766 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-5e74f6ab-7feb-44ee-8475-79b96995badf None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:16:27.119965 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5e74f6ab-7feb-44ee-8475-79b96995badf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:16:27.213496 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 746/1486] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:16:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:16:27.251606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4cd998-fa7a-4515-8491-c0b7c8c39e58 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:16:27.253120 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2e4cd998-fa7a-4515-8491-c0b7c8c39e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:16:27.357735 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4cd998-fa7a-4515-8491-c0b7c8c39e58 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:16:27.429206 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:16:27.431408 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5178c2f-6430-480c-b392-12c669041268 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:16:27.432255 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5178c2f-6430-480c-b392-12c669041268 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:16:27.432712 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5178c2f-6430-480c-b392-12c669041268 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:16:27.433207 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5178c2f-6430-480c-b392-12c669041268 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:16:27.434883 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d5178c2f-6430-480c-b392-12c669041268 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:16:27.546046 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5178c2f-6430-480c-b392-12c669041268 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:16:27.547767 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5178c2f-6430-480c-b392-12c669041268 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:16:27.566098 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 741/1487] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:16:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:16:29.691498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d548b9-6339-42c1-87d9-4ebc524c259b None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:16:29.692061 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d548b9-6339-42c1-87d9-4ebc524c259b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:16:29.692061 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d548b9-6339-42c1-87d9-4ebc524c259b None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:16:29.693459 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 747/1488] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:16:29.699145 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e6c1e-07bc-4327-9dad-b0381a34f341 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:16:29.699587 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e6c1e-07bc-4327-9dad-b0381a34f341 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:16:29.699688 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e6c1e-07bc-4327-9dad-b0381a34f341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:16:29.766036 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-bd1e6c1e-07bc-4327-9dad-b0381a34f341 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:16:29.767185 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-bd1e6c1e-07bc-4327-9dad-b0381a34f341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:16:29.821808 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:16:29.867821 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 742/1489] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:16:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:16:29.908845 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d548b9-6339-42c1-87d9-4ebc524c259b None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:16:29.910422 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-49d548b9-6339-42c1-87d9-4ebc524c259b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:16:29.948080 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:16:30.016379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d548b9-6339-42c1-87d9-4ebc524c259b None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:16:30.089632 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eef898-f8a9-4de7-9b1e-480af42d384d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:16:30.090280 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eef898-f8a9-4de7-9b1e-480af42d384d service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:16:30.090559 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eef898-f8a9-4de7-9b1e-480af42d384d service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:16:30.090828 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eef898-f8a9-4de7-9b1e-480af42d384d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:16:30.092480 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-11eef898-f8a9-4de7-9b1e-480af42d384d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:16:30.172544 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:16:30.203989 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eef898-f8a9-4de7-9b1e-480af42d384d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:16:30.205428 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eef898-f8a9-4de7-9b1e-480af42d384d service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:16:30.225081 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 748/1490] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:16:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:16:32.400175 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95d753-229c-4060-b527-ef17d319a703 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:16:32.400755 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95d753-229c-4060-b527-ef17d319a703 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:16:32.400755 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95d753-229c-4060-b527-ef17d319a703 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:16:32.402142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 743/1491] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 27 04:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:16:32.407445 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb6137-6269-4966-8278-8eaf15a0a8eb None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:16:32.407904 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb6137-6269-4966-8278-8eaf15a0a8eb None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:16:32.408009 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb6137-6269-4966-8278-8eaf15a0a8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:16:32.473212 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-42cb6137-6269-4966-8278-8eaf15a0a8eb None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:16:32.474459 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-42cb6137-6269-4966-8278-8eaf15a0a8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:16:32.572187 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 749/1492] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 27 04:16:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:16:32.611058 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95d753-229c-4060-b527-ef17d319a703 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:16:32.612576 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0b95d753-229c-4060-b527-ef17d319a703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:16:32.717497 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95d753-229c-4060-b527-ef17d319a703 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:16:32.795295 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7db8be-31eb-4e2b-a83c-80c5c409911a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:16:32.795912 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7db8be-31eb-4e2b-a83c-80c5c409911a service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:16:32.796190 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7db8be-31eb-4e2b-a83c-80c5c409911a service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:16:32.796459 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7db8be-31eb-4e2b-a83c-80c5c409911a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:16:32.798110 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-1c7db8be-31eb-4e2b-a83c-80c5c409911a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:16:32.807595 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:16:32.909524 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7db8be-31eb-4e2b-a83c-80c5c409911a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:16:32.910981 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7db8be-31eb-4e2b-a83c-80c5c409911a service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:16:32.930090 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 744/1493] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:16:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:16:42.570402 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=82 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:16:42.926757 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=50 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:18:01.760076 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa10528-ad59-4d65-82b9-11c9b81d54f8 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:18:01.760512 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa10528-ad59-4d65-82b9-11c9b81d54f8 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:18:01.760640 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa10528-ad59-4d65-82b9-11c9b81d54f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:18:01.837399 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-dfa10528-ad59-4d65-82b9-11c9b81d54f8 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:18:01.838656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-dfa10528-ad59-4d65-82b9-11c9b81d54f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:18:01.935315 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 750/1494] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:18:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:18:01.943898 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5769b0-24f9-4d08-97e0-1a8892ce55f5 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:18:01.944487 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5769b0-24f9-4d08-97e0-1a8892ce55f5 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:18:01.944487 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5769b0-24f9-4d08-97e0-1a8892ce55f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:18:02.020675 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-ab5769b0-24f9-4d08-97e0-1a8892ce55f5 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:18:02.021953 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ab5769b0-24f9-4d08-97e0-1a8892ce55f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:18:02.116916 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 745/1495] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:18:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:18:02.121615 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa10528-ad59-4d65-82b9-11c9b81d54f8 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:18:02.122863 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-dfa10528-ad59-4d65-82b9-11c9b81d54f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:18:02.134462 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa10528-ad59-4d65-82b9-11c9b81d54f8 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:18:02.197226 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:18:02.238530 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b214c2f5-094a-45f1-b323-68840c2a6ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:18:02.239152 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214c2f5-094a-45f1-b323-68840c2a6ace admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:18:02.239430 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214c2f5-094a-45f1-b323-68840c2a6ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:18:02.239702 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214c2f5-094a-45f1-b323-68840c2a6ace admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:18:02.240552 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b214c2f5-094a-45f1-b323-68840c2a6ace admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:18:02.240786 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b214c2f5-094a-45f1-b323-68840c2a6ace admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:18:02.241791 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: WARNING py.warnings [None req-b214c2f5-094a-45f1-b323-68840c2a6ace admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 27 04:18:02.241791 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: warnings.warn(msg) Jul 27 04:18:02.241791 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: Jul 27 04:18:02.242693 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b214c2f5-094a-45f1-b323-68840c2a6ace admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:18:02.253480 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 751/1496] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:18:02.261757 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5769b0-24f9-4d08-97e0-1a8892ce55f5 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:18:02.270905 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5769b0-24f9-4d08-97e0-1a8892ce55f5 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:18:02.338853 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:18:02.372616 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a0083-475f-4992-bc47-71c7a1bad5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:18:02.373251 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a0083-475f-4992-bc47-71c7a1bad5c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:18:02.373533 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a0083-475f-4992-bc47-71c7a1bad5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:18:02.373807 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a0083-475f-4992-bc47-71c7a1bad5c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:18:02.374520 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98a0083-475f-4992-bc47-71c7a1bad5c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:18:02.376302 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: WARNING py.warnings [None req-e98a0083-475f-4992-bc47-71c7a1bad5c8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 04:18:02.376302 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: warnings.warn(msg) Jul 27 04:18:02.376302 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: Jul 27 04:18:02.377501 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98a0083-475f-4992-bc47-71c7a1bad5c8 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:18:02.434230 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 746/1497] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:18:02 2022] POST /identity/v3/projects => generated 377 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:18:02.438751 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b214c2f5-094a-45f1-b323-68840c2a6ace admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:18:02.440136 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b214c2f5-094a-45f1-b323-68840c2a6ace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:18:02.475989 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b214c2f5-094a-45f1-b323-68840c2a6ace admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:18:02.571435 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e34c2-66cb-4a43-9e60-0e1c53a5dd61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:18:02.571963 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e34c2-66cb-4a43-9e60-0e1c53a5dd61 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:18:02.572238 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e34c2-66cb-4a43-9e60-0e1c53a5dd61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:18:02.572507 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e34c2-66cb-4a43-9e60-0e1c53a5dd61 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:18:02.573162 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e34c2-66cb-4a43-9e60-0e1c53a5dd61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:18:02.574183 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: WARNING py.warnings [None req-8f4e34c2-66cb-4a43-9e60-0e1c53a5dd61 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 04:18:02.574183 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: warnings.warn(msg) Jul 27 04:18:02.574183 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: Jul 27 04:18:02.575380 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e34c2-66cb-4a43-9e60-0e1c53a5dd61 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:18:02.616759 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 752/1498] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:18:02 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:18:02.620985 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a0083-475f-4992-bc47-71c7a1bad5c8 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:18:02.622555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e98a0083-475f-4992-bc47-71c7a1bad5c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:18:02.692633 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a0083-475f-4992-bc47-71c7a1bad5c8 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:18:02.777144 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:18:02.788980 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82317ff3-d9b1-417f-a37a-6b4b2c941b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:18:02.789608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82317ff3-d9b1-417f-a37a-6b4b2c941b99 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:18:02.789889 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82317ff3-d9b1-417f-a37a-6b4b2c941b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:18:02.790191 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82317ff3-d9b1-417f-a37a-6b4b2c941b99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:18:02.790887 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82317ff3-d9b1-417f-a37a-6b4b2c941b99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:18:02.791892 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: WARNING py.warnings [None req-82317ff3-d9b1-417f-a37a-6b4b2c941b99 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 27 04:18:02.791892 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: warnings.warn(msg) Jul 27 04:18:02.791892 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: Jul 27 04:18:02.792780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82317ff3-d9b1-417f-a37a-6b4b2c941b99 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:18:02.806233 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 747/1499] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:18:02.810394 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e34c2-66cb-4a43-9e60-0e1c53a5dd61 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:18:02.811929 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8f4e34c2-66cb-4a43-9e60-0e1c53a5dd61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:18:02.823968 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e34c2-66cb-4a43-9e60-0e1c53a5dd61 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:18:02.849258 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:18:02.918476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba6cc14-4f09-401b-ac0d-585440f4513d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:18:02.919100 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba6cc14-4f09-401b-ac0d-585440f4513d admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:18:02.919377 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba6cc14-4f09-401b-ac0d-585440f4513d admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:18:02.919653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba6cc14-4f09-401b-ac0d-585440f4513d admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:18:02.920343 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba6cc14-4f09-401b-ac0d-585440f4513d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:18:02.921348 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: WARNING py.warnings [None req-cba6cc14-4f09-401b-ac0d-585440f4513d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 27 04:18:02.921348 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: warnings.warn(msg) Jul 27 04:18:02.921348 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: Jul 27 04:18:02.922254 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba6cc14-4f09-401b-ac0d-585440f4513d admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:18:02.935187 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 753/1500] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:18:02.939473 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82317ff3-d9b1-417f-a37a-6b4b2c941b99 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:18:02.941033 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-82317ff3-d9b1-417f-a37a-6b4b2c941b99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:18:02.977988 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82317ff3-d9b1-417f-a37a-6b4b2c941b99 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:18:03.075227 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063de11d-e7c4-496c-b1ab-7df4610cfeb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:18:03.075813 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063de11d-e7c4-496c-b1ab-7df4610cfeb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:18:03.076090 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063de11d-e7c4-496c-b1ab-7df4610cfeb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:18:03.076361 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063de11d-e7c4-496c-b1ab-7df4610cfeb9 admin admin] PATH_INFO: `/v3/projects/738382499d9d4bf69add0be8f0605fde/users/53aa8d1318d24138b2f51b14c4efd8a8/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:18:03.098619 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063de11d-e7c4-496c-b1ab-7df4610cfeb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=738382499d9d4bf69add0be8f0605fde, user_id=53aa8d1318d24138b2f51b14c4efd8a8, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:18:03.099693 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: WARNING py.warnings [None req-063de11d-e7c4-496c-b1ab-7df4610cfeb9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 04:18:03.099693 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: warnings.warn(msg) Jul 27 04:18:03.099693 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: Jul 27 04:18:03.101623 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063de11d-e7c4-496c-b1ab-7df4610cfeb9 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:18:03.127202 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 748/1501] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:18:02 2022] PUT /identity/v3/projects/738382499d9d4bf69add0be8f0605fde/users/53aa8d1318d24138b2f51b14c4efd8a8/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:18:03.140096 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195160f8-f36f-4e68-96dd-847b1488ac87 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:18:03.140385 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195160f8-f36f-4e68-96dd-847b1488ac87 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:18:03.140665 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195160f8-f36f-4e68-96dd-847b1488ac87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:18:03.198033 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-195160f8-f36f-4e68-96dd-847b1488ac87 None None] MFA Rules not processed for user `53aa8d1318d24138b2f51b14c4efd8a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:18:03.199246 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-195160f8-f36f-4e68-96dd-847b1488ac87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:18:03.274288 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 754/1502] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:18:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:18:03.288047 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8883e3-9fd6-4fca-b9ff-999248ca60ea None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:18:03.288341 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8883e3-9fd6-4fca-b9ff-999248ca60ea None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:18:03.288616 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8883e3-9fd6-4fca-b9ff-999248ca60ea None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:18:03.289961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 749/1503] 10.222.0.46 () {54 vars in 964 bytes} [Wed Jul 27 04:18:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:18:03.294733 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd18f4-67a6-43e4-996e-dbc99201c314 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:18:03.295016 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd18f4-67a6-43e4-996e-dbc99201c314 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:18:03.295290 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd18f4-67a6-43e4-996e-dbc99201c314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:18:03.334591 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:18:03.383296 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-b2cd18f4-67a6-43e4-996e-dbc99201c314 None None] MFA Rules not processed for user `134ef452d7814c73b7c371a1cdbcbc00`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:18:03.384519 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b2cd18f4-67a6-43e4-996e-dbc99201c314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:18:03.458522 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 755/1504] 10.222.0.46 () {58 vars in 1061 bytes} [Wed Jul 27 04:18:03 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:18:03.466398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8883e3-9fd6-4fca-b9ff-999248ca60ea None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:18:03.467901 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ae8883e3-9fd6-4fca-b9ff-999248ca60ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:18:03.521050 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:18:03.575976 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8883e3-9fd6-4fca-b9ff-999248ca60ea None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:18:03.650449 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7743d7d-36e8-4405-924f-25a2f2e55bdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:18:03.650992 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7743d7d-36e8-4405-924f-25a2f2e55bdc service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:18:03.651275 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7743d7d-36e8-4405-924f-25a2f2e55bdc service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:18:03.651550 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7743d7d-36e8-4405-924f-25a2f2e55bdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:18:03.653178 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e7743d7d-36e8-4405-924f-25a2f2e55bdc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:18:03.752420 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7743d7d-36e8-4405-924f-25a2f2e55bdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:18:03.753898 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7743d7d-36e8-4405-924f-25a2f2e55bdc service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:18:03.759237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:18:03.775876 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 750/1505] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:18:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 04:18:03.822486 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a39c64-d070-4c2d-9fd5-e8a9a3279aab None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:18:03.822937 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a39c64-d070-4c2d-9fd5-e8a9a3279aab None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:18:03.823052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a39c64-d070-4c2d-9fd5-e8a9a3279aab None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:18:03.824322 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 756/1506] 10.222.0.46 () {52 vars in 830 bytes} [Wed Jul 27 04:18:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:18:03.827988 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7743d7d-36e8-4405-924f-25a2f2e55bdc service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:18:03.829210 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e7743d7d-36e8-4405-924f-25a2f2e55bdc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:18:03.933644 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7743d7d-36e8-4405-924f-25a2f2e55bdc service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:18:04.027774 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6942cca-3cc1-40f5-a664-d45949600c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:18:04.028486 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6942cca-3cc1-40f5-a664-d45949600c7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:18:04.028689 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6942cca-3cc1-40f5-a664-d45949600c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:18:04.028964 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6942cca-3cc1-40f5-a664-d45949600c7a admin admin] PATH_INFO: `/v3/users/53aa8d1318d24138b2f51b14c4efd8a8` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:18:04.041668 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6942cca-3cc1-40f5-a664-d45949600c7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=53aa8d1318d24138b2f51b14c4efd8a8)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:18:04.042755 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: WARNING py.warnings [None req-b6942cca-3cc1-40f5-a664-d45949600c7a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 04:18:04.042755 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: warnings.warn(msg) Jul 27 04:18:04.042755 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: Jul 27 04:18:04.043986 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6942cca-3cc1-40f5-a664-d45949600c7a admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:18:04.054100 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:18:04.106297 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-b6942cca-3cc1-40f5-a664-d45949600c7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53aa8d1318d24138b2f51b14c4efd8a8'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:18:04.106872 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-b6942cca-3cc1-40f5-a664-d45949600c7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53aa8d1318d24138b2f51b14c4efd8a8'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:18:04.116100 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-b6942cca-3cc1-40f5-a664-d45949600c7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53aa8d1318d24138b2f51b14c4efd8a8'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:18:04.134625 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-b6942cca-3cc1-40f5-a664-d45949600c7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53aa8d1318d24138b2f51b14c4efd8a8'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:18:04.147495 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-b6942cca-3cc1-40f5-a664-d45949600c7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53aa8d1318d24138b2f51b14c4efd8a8'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:18:04.153098 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 751/1507] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:18:03 2022] DELETE /identity/v3/users/53aa8d1318d24138b2f51b14c4efd8a8 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:18:04.157455 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa35dea-ca5a-47ad-8e65-c28390b62158 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:18:04.157811 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa35dea-ca5a-47ad-8e65-c28390b62158 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:18:04.158179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa35dea-ca5a-47ad-8e65-c28390b62158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:18:04.215416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-caa35dea-ca5a-47ad-8e65-c28390b62158 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:18:04.216529 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-caa35dea-ca5a-47ad-8e65-c28390b62158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:18:04.292007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:18:04.305957 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 757/1508] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:18:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:18:04.316213 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6942cca-3cc1-40f5-a664-d45949600c7a admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:18:04.317712 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b6942cca-3cc1-40f5-a664-d45949600c7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:18:04.418414 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6942cca-3cc1-40f5-a664-d45949600c7a admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:18:04.472459 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:18:04.494037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d422d-ea42-4e00-af57-02b749880416 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:18:04.494669 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d422d-ea42-4e00-af57-02b749880416 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:18:04.494940 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d422d-ea42-4e00-af57-02b749880416 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:18:04.495206 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d422d-ea42-4e00-af57-02b749880416 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:18:04.496821 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e18d422d-ea42-4e00-af57-02b749880416 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:18:04.612201 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d422d-ea42-4e00-af57-02b749880416 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:18:04.613652 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d422d-ea42-4e00-af57-02b749880416 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:18:04.631775 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 752/1509] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:18:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:18:04.996579 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa35dea-ca5a-47ad-8e65-c28390b62158 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:18:04.997837 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-caa35dea-ca5a-47ad-8e65-c28390b62158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:18:05.009965 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa35dea-ca5a-47ad-8e65-c28390b62158 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:18:05.107712 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79249cb5-c01f-4f54-b516-6c038090afe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:18:05.108360 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79249cb5-c01f-4f54-b516-6c038090afe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:18:05.108638 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79249cb5-c01f-4f54-b516-6c038090afe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:18:05.108910 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79249cb5-c01f-4f54-b516-6c038090afe0 admin admin] PATH_INFO: `/v3/projects/738382499d9d4bf69add0be8f0605fde` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:18:05.119120 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79249cb5-c01f-4f54-b516-6c038090afe0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=738382499d9d4bf69add0be8f0605fde)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:18:05.120140 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: WARNING py.warnings [None req-79249cb5-c01f-4f54-b516-6c038090afe0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 04:18:05.120140 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: warnings.warn(msg) Jul 27 04:18:05.120140 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: Jul 27 04:18:05.121320 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79249cb5-c01f-4f54-b516-6c038090afe0 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:18:05.176909 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-79249cb5-c01f-4f54-b516-6c038090afe0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '738382499d9d4bf69add0be8f0605fde', 'request_id': 'req-79249cb5-c01f-4f54-b516-6c038090afe0'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:18:05.190810 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-79249cb5-c01f-4f54-b516-6c038090afe0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '738382499d9d4bf69add0be8f0605fde', 'request_id': 'req-79249cb5-c01f-4f54-b516-6c038090afe0'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:18:05.197615 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-79249cb5-c01f-4f54-b516-6c038090afe0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '738382499d9d4bf69add0be8f0605fde', 'request_id': 'req-79249cb5-c01f-4f54-b516-6c038090afe0'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:18:05.226135 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-79249cb5-c01f-4f54-b516-6c038090afe0 admin admin] The token cache is being invalidate because project 738382499d9d4bf69add0be8f0605fde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:18:05.226434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-79249cb5-c01f-4f54-b516-6c038090afe0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:18:05.226870 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-79249cb5-c01f-4f54-b516-6c038090afe0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:18:05.228357 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 758/1510] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:18:04 2022] DELETE /identity/v3/projects/738382499d9d4bf69add0be8f0605fde => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:18:14.629412 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=62 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:18:15.223676 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:19:39.841752 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedfb38-e3fb-4dfe-93cd-a4b105e14447 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:19:39.842411 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedfb38-e3fb-4dfe-93cd-a4b105e14447 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:19:39.842411 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedfb38-e3fb-4dfe-93cd-a4b105e14447 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:19:39.843687 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 04:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:19:39.848523 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e46da-15ee-4cc2-9dcb-0875bc64d825 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:19:39.849190 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e46da-15ee-4cc2-9dcb-0875bc64d825 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:19:39.849465 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e46da-15ee-4cc2-9dcb-0875bc64d825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:19:39.936493 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-2b5e46da-15ee-4cc2-9dcb-0875bc64d825 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:19:39.937790 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2b5e46da-15ee-4cc2-9dcb-0875bc64d825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:19:40.044054 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 04:19:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:19:40.196477 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedfb38-e3fb-4dfe-93cd-a4b105e14447 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:19:40.198858 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-6bedfb38-e3fb-4dfe-93cd-a4b105e14447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:19:40.316744 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedfb38-e3fb-4dfe-93cd-a4b105e14447 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:19:40.323175 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959069be-7aca-4832-9ab2-c184b81c15c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:19:40.323803 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959069be-7aca-4832-9ab2-c184b81c15c1 service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:19:40.324088 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959069be-7aca-4832-9ab2-c184b81c15c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:19:40.324365 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959069be-7aca-4832-9ab2-c184b81c15c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:19:40.332372 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959069be-7aca-4832-9ab2-c184b81c15c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:19:40.333956 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959069be-7aca-4832-9ab2-c184b81c15c1 service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:19:40.343559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 754/1513] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:19:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:19:41.199285 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055da67-8ed3-4645-8901-46f40ba70f99 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:19:41.199834 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055da67-8ed3-4645-8901-46f40ba70f99 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:19:41.199834 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055da67-8ed3-4645-8901-46f40ba70f99 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:19:41.201122 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 760/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 04:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:19:41.205921 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c29ce7-1bb7-4747-98b9-376b8ea9826b None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:19:41.206370 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c29ce7-1bb7-4747-98b9-376b8ea9826b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:19:41.206507 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c29ce7-1bb7-4747-98b9-376b8ea9826b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:19:41.227668 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-13c29ce7-1bb7-4747-98b9-376b8ea9826b None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:19:41.228852 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-13c29ce7-1bb7-4747-98b9-376b8ea9826b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:19:41.236800 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 755/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 04:19:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:19:41.389063 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6055da67-8ed3-4645-8901-46f40ba70f99 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:19:41.403693 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6055da67-8ed3-4645-8901-46f40ba70f99 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:19:41.409147 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e22e7-5ffe-4176-8c53-b48b915af3d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:19:41.409763 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e22e7-5ffe-4176-8c53-b48b915af3d8 service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:19:41.410078 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e22e7-5ffe-4176-8c53-b48b915af3d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:19:41.410361 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e22e7-5ffe-4176-8c53-b48b915af3d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:19:41.419363 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e22e7-5ffe-4176-8c53-b48b915af3d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:19:41.420943 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e22e7-5ffe-4176-8c53-b48b915af3d8 service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:19:41.430886 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 761/1516] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:19:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:19:42.038627 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40f270-f07e-4069-8b10-bc4ad5f8292f None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:19:42.038914 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40f270-f07e-4069-8b10-bc4ad5f8292f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:19:42.039197 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40f270-f07e-4069-8b10-bc4ad5f8292f None None] PATH_INFO: `/v3` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:19:42.040496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 756/1517] 10.222.0.27 () {54 vars in 925 bytes} [Wed Jul 27 04:19:42 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:19:42.046875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314fa8b-46bd-47db-b091-985a3c48359c None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:19:42.047168 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314fa8b-46bd-47db-b091-985a3c48359c None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:19:42.047452 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314fa8b-46bd-47db-b091-985a3c48359c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:19:42.105938 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-1314fa8b-46bd-47db-b091-985a3c48359c None None] MFA Rules not processed for user `8c08fba2d7de42218400c5e652cc3de0`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:19:42.107175 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-1314fa8b-46bd-47db-b091-985a3c48359c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:19:42.124420 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:19:42.178652 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 762/1518] 10.222.0.27 () {58 vars in 1015 bytes} [Wed Jul 27 04:19:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:19:45.970063 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1cec2-5248-4b9c-83cd-79fd9fa9e5ae None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:19:45.970605 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1cec2-5248-4b9c-83cd-79fd9fa9e5ae None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:19:45.970605 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1cec2-5248-4b9c-83cd-79fd9fa9e5ae None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:19:45.971935 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 757/1519] 10.222.0.46 () {54 vars in 961 bytes} [Wed Jul 27 04:19:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:19:45.977034 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722dc71-efc2-47a0-96cc-4ec35968c943 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:19:45.977416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722dc71-efc2-47a0-96cc-4ec35968c943 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:19:45.977599 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722dc71-efc2-47a0-96cc-4ec35968c943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:19:45.998057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-7722dc71-efc2-47a0-96cc-4ec35968c943 None None] MFA Rules not processed for user `8c08fba2d7de42218400c5e652cc3de0`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:19:45.999192 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7722dc71-efc2-47a0-96cc-4ec35968c943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:19:46.007855 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 763/1520] 10.222.0.46 () {58 vars in 1058 bytes} [Wed Jul 27 04:19:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:19:46.015556 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc1cec2-5248-4b9c-83cd-79fd9fa9e5ae None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:19:46.028806 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc1cec2-5248-4b9c-83cd-79fd9fa9e5ae None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:19:46.033622 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e072bc-51b1-43dd-bdcb-02ad6ca4004a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c08fba2d7de42218400c5e652cc3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:19:46.034260 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e072bc-51b1-43dd-bdcb-02ad6ca4004a service placement] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:19:46.034545 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e072bc-51b1-43dd-bdcb-02ad6ca4004a service placement] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:19:46.034816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e072bc-51b1-43dd-bdcb-02ad6ca4004a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:19:46.042907 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e072bc-51b1-43dd-bdcb-02ad6ca4004a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:19:46.045596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e072bc-51b1-43dd-bdcb-02ad6ca4004a service placement] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:19:46.055194 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 758/1521] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:19:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:19:52.660662 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5286e2-f822-45e4-8fc7-53c7bf585cff None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:19:52.661229 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5286e2-f822-45e4-8fc7-53c7bf585cff None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:19:52.661229 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5286e2-f822-45e4-8fc7-53c7bf585cff None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:19:52.662622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 04:19:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:19:52.667378 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a1f93-d649-4359-8f2a-d9db17b930eb None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:19:52.667798 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a1f93-d649-4359-8f2a-d9db17b930eb None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:19:52.668464 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a1f93-d649-4359-8f2a-d9db17b930eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:19:52.688081 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-2e1a1f93-d649-4359-8f2a-d9db17b930eb None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:19:52.689284 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2e1a1f93-d649-4359-8f2a-d9db17b930eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:19:52.697352 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 04:19:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:19:52.852088 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5286e2-f822-45e4-8fc7-53c7bf585cff None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:19:52.867232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5286e2-f822-45e4-8fc7-53c7bf585cff None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:19:52.873281 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11cae4-8126-45fe-b335-21f39ba054dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:19:52.873881 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11cae4-8126-45fe-b335-21f39ba054dd service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:19:52.874187 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11cae4-8126-45fe-b335-21f39ba054dd service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:19:52.874459 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11cae4-8126-45fe-b335-21f39ba054dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:19:52.882429 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa11cae4-8126-45fe-b335-21f39ba054dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:19:52.884000 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa11cae4-8126-45fe-b335-21f39ba054dd service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:19:52.893220 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 765/1524] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:19:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:19:53.711546 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f1b99-c757-4288-a9f9-928022c062a2 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:19:53.712060 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f1b99-c757-4288-a9f9-928022c062a2 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:19:53.712060 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f1b99-c757-4288-a9f9-928022c062a2 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:19:53.713415 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 04:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:19:53.718820 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd1a08-100d-44e9-ba82-066e558fccba None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:19:53.719221 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd1a08-100d-44e9-ba82-066e558fccba None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:19:53.719376 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd1a08-100d-44e9-ba82-066e558fccba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:19:53.738723 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-e9cd1a08-100d-44e9-ba82-066e558fccba None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:19:53.739909 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e9cd1a08-100d-44e9-ba82-066e558fccba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:19:53.747706 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 04:19:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:19:53.900479 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f1b99-c757-4288-a9f9-928022c062a2 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:19:53.914869 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f1b99-c757-4288-a9f9-928022c062a2 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:19:53.920354 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bdbed2-4eba-451f-9263-1800266a7961 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:19:53.920961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bdbed2-4eba-451f-9263-1800266a7961 service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:19:53.921242 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bdbed2-4eba-451f-9263-1800266a7961 service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:19:53.921516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bdbed2-4eba-451f-9263-1800266a7961 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:19:53.929515 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bdbed2-4eba-451f-9263-1800266a7961 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:19:53.931121 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bdbed2-4eba-451f-9263-1800266a7961 service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:19:53.941188 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 761/1527] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:19:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:19:57.708675 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0e4b0-ec5f-477b-a2c4-9d7aeb13651e None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:19:57.709237 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0e4b0-ec5f-477b-a2c4-9d7aeb13651e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:19:57.709237 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0e4b0-ec5f-477b-a2c4-9d7aeb13651e None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:19:57.710610 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 04:19:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:19:57.716761 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a19590-39f6-4f2e-ae98-34f3e8a63505 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:19:57.717201 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a19590-39f6-4f2e-ae98-34f3e8a63505 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:19:57.717298 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a19590-39f6-4f2e-ae98-34f3e8a63505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:19:57.736795 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-81a19590-39f6-4f2e-ae98-34f3e8a63505 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:19:57.738002 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-81a19590-39f6-4f2e-ae98-34f3e8a63505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:19:57.745717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 04:19:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:19:57.821283 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0e4b0-ec5f-477b-a2c4-9d7aeb13651e None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:19:57.823049 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5dc0e4b0-ec5f-477b-a2c4-9d7aeb13651e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:19:57.900728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0e4b0-ec5f-477b-a2c4-9d7aeb13651e None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:19:57.904971 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779f9c4-345a-446d-80e4-2a63feb8ccd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:19:57.905595 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779f9c4-345a-446d-80e4-2a63feb8ccd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:19:57.905879 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779f9c4-345a-446d-80e4-2a63feb8ccd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:19:57.906187 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779f9c4-345a-446d-80e4-2a63feb8ccd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:19:57.915335 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e779f9c4-345a-446d-80e4-2a63feb8ccd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:19:57.916761 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e779f9c4-345a-446d-80e4-2a63feb8ccd3 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:19:57.926209 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 768/1530] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:19:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:19:59.960474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9656afa-b498-4940-95e6-6de86c568510 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:19:59.961023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9656afa-b498-4940-95e6-6de86c568510 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:19:59.961023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9656afa-b498-4940-95e6-6de86c568510 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:19:59.962465 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 04:19:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:19:59.968388 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff5990-cd51-44f7-8ef9-a9ca5a44f029 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:19:59.968844 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff5990-cd51-44f7-8ef9-a9ca5a44f029 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:19:59.968944 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff5990-cd51-44f7-8ef9-a9ca5a44f029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:19:59.989538 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-02ff5990-cd51-44f7-8ef9-a9ca5a44f029 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:19:59.990782 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-02ff5990-cd51-44f7-8ef9-a9ca5a44f029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:19:59.999340 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 04:19:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:20:00.036530 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9656afa-b498-4940-95e6-6de86c568510 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:20:00.038241 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-a9656afa-b498-4940-95e6-6de86c568510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:20:00.052146 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9656afa-b498-4940-95e6-6de86c568510 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:20:00.055603 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf11b363-2046-4f57-95fd-45fac4635102 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:20:00.056202 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11b363-2046-4f57-95fd-45fac4635102 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:20:00.056479 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11b363-2046-4f57-95fd-45fac4635102 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:20:00.056748 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11b363-2046-4f57-95fd-45fac4635102 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:20:00.066444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf11b363-2046-4f57-95fd-45fac4635102 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:20:00.067804 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf11b363-2046-4f57-95fd-45fac4635102 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:20:00.077758 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 764/1533] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:20:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:20:03.579729 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5e04a-4638-4bc6-8db5-2b6bfb7d2213 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:20:03.580311 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5e04a-4638-4bc6-8db5-2b6bfb7d2213 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:20:03.580311 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5e04a-4638-4bc6-8db5-2b6bfb7d2213 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:20:03.581633 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 04:20:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:20:03.586713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b78b0-30a3-4fed-8ce5-f93e9b20b4a7 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:20:03.587168 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b78b0-30a3-4fed-8ce5-f93e9b20b4a7 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:20:03.587278 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b78b0-30a3-4fed-8ce5-f93e9b20b4a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:20:03.608003 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-909b78b0-30a3-4fed-8ce5-f93e9b20b4a7 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:20:03.609200 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-909b78b0-30a3-4fed-8ce5-f93e9b20b4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:20:03.617002 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 04:20:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:20:03.926828 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f5e04a-4638-4bc6-8db5-2b6bfb7d2213 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:20:03.928541 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b7f5e04a-4638-4bc6-8db5-2b6bfb7d2213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:20:03.992951 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:20:04.007586 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f5e04a-4638-4bc6-8db5-2b6bfb7d2213 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:20:04.012144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d760ab9-46f3-4bbe-9093-969824d6ab85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:20:04.013103 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d760ab9-46f3-4bbe-9093-969824d6ab85 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:20:04.013382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d760ab9-46f3-4bbe-9093-969824d6ab85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:20:04.013660 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d760ab9-46f3-4bbe-9093-969824d6ab85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:20:04.022905 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d760ab9-46f3-4bbe-9093-969824d6ab85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:20:04.024329 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d760ab9-46f3-4bbe-9093-969824d6ab85 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:20:04.034671 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 771/1536] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:20:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:20:06.112788 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeedb0a-be66-4786-b9fc-33e8dfb46198 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:20:06.113401 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeedb0a-be66-4786-b9fc-33e8dfb46198 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:20:06.113520 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeedb0a-be66-4786-b9fc-33e8dfb46198 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:20:06.115026 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 04:20:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:20:06.120585 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb12fc1-9df1-4132-af57-07c9feac0f0d None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:20:06.121242 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb12fc1-9df1-4132-af57-07c9feac0f0d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:20:06.121242 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb12fc1-9df1-4132-af57-07c9feac0f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:20:06.140599 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-beb12fc1-9df1-4132-af57-07c9feac0f0d None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:20:06.141813 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-beb12fc1-9df1-4132-af57-07c9feac0f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:20:06.149938 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 04:20:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:20:06.452328 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeedb0a-be66-4786-b9fc-33e8dfb46198 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:20:06.454109 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-bbeedb0a-be66-4786-b9fc-33e8dfb46198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:20:06.461664 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=84 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:20:06.469812 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeedb0a-be66-4786-b9fc-33e8dfb46198 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:20:06.473389 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757e083-7a41-491a-bc3b-148383b11b0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:20:06.474043 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757e083-7a41-491a-bc3b-148383b11b0f service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:20:06.474333 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757e083-7a41-491a-bc3b-148383b11b0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:20:06.474607 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757e083-7a41-491a-bc3b-148383b11b0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:20:06.484010 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7757e083-7a41-491a-bc3b-148383b11b0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:20:06.485452 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7757e083-7a41-491a-bc3b-148383b11b0f service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:20:06.494836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 767/1539] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:20:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:20:16.135331 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=11 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:20:16.490642 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=5 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:34.017083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852fcfd-fb88-49b8-b859-9e96aac58411 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:34.017755 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852fcfd-fb88-49b8-b859-9e96aac58411 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:34.017755 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852fcfd-fb88-49b8-b859-9e96aac58411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:34.042578 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-0852fcfd-fb88-49b8-b859-9e96aac58411 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:34.043792 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-0852fcfd-fb88-49b8-b859-9e96aac58411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:34.055833 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 773/1540] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:34.065158 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec058d3-dd63-4e53-a5e1-53b6618ae1a6 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:34.065723 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec058d3-dd63-4e53-a5e1-53b6618ae1a6 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:34.065723 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec058d3-dd63-4e53-a5e1-53b6618ae1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:34.093250 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-dec058d3-dd63-4e53-a5e1-53b6618ae1a6 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:34.094519 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-dec058d3-dd63-4e53-a5e1-53b6618ae1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:34.106467 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 768/1541] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:34.111095 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852fcfd-fb88-49b8-b859-9e96aac58411 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:34.125644 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852fcfd-fb88-49b8-b859-9e96aac58411 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:34.134359 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec14adb-ab16-47b7-b45f-fc70a3ba0bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:34.134999 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec14adb-ab16-47b7-b45f-fc70a3ba0bfc admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:34.135914 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec14adb-ab16-47b7-b45f-fc70a3ba0bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:34.136109 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec14adb-ab16-47b7-b45f-fc70a3ba0bfc admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:34.136826 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec14adb-ab16-47b7-b45f-fc70a3ba0bfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:21:34.137128 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec14adb-ab16-47b7-b45f-fc70a3ba0bfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:34.138808 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec14adb-ab16-47b7-b45f-fc70a3ba0bfc admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:34.156054 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 774/1542] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:21:34.164102 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec058d3-dd63-4e53-a5e1-53b6618ae1a6 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:34.178087 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec058d3-dd63-4e53-a5e1-53b6618ae1a6 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:34.184563 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06dbd9b-1265-407a-a08c-6aae76fa1839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:34.185190 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06dbd9b-1265-407a-a08c-6aae76fa1839 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:34.185466 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06dbd9b-1265-407a-a08c-6aae76fa1839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:34.185736 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06dbd9b-1265-407a-a08c-6aae76fa1839 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:34.186490 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06dbd9b-1265-407a-a08c-6aae76fa1839 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:34.188390 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06dbd9b-1265-407a-a08c-6aae76fa1839 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:34.246160 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb307ce1-6d5c-4f39-8727-80b377b3ff79 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:34.246403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb307ce1-6d5c-4f39-8727-80b377b3ff79 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:34.246656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb307ce1-6d5c-4f39-8727-80b377b3ff79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:34.251351 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 769/1543] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:21:34 2022] POST /identity/v3/projects => generated 391 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:34.255215 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06dbd9b-1265-407a-a08c-6aae76fa1839 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:34.285242 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-cb307ce1-6d5c-4f39-8727-80b377b3ff79 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:34.286592 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-cb307ce1-6d5c-4f39-8727-80b377b3ff79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:34.357037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06dbd9b-1265-407a-a08c-6aae76fa1839 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:34.365656 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe1f6e-2904-4feb-928a-8d1fa3ddbf8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:34.368890 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe1f6e-2904-4feb-928a-8d1fa3ddbf8e admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:34.369430 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe1f6e-2904-4feb-928a-8d1fa3ddbf8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:34.370807 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe1f6e-2904-4feb-928a-8d1fa3ddbf8e admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:34.371748 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffe1f6e-2904-4feb-928a-8d1fa3ddbf8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:34.373011 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: WARNING py.warnings [None req-2ffe1f6e-2904-4feb-928a-8d1fa3ddbf8e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 04:21:34.373011 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: warnings.warn(msg) Jul 27 04:21:34.373011 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: Jul 27 04:21:34.374671 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffe1f6e-2904-4feb-928a-8d1fa3ddbf8e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:34.407662 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 775/1544] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 27 04:21:34.408162 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb87120e-5c9f-4985-b7e1-e81294e80fcf None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:34.408454 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb87120e-5c9f-4985-b7e1-e81294e80fcf None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:34.408724 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb87120e-5c9f-4985-b7e1-e81294e80fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:34.418016 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 770/1545] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:21:34 2022] POST /identity/v3/users => generated 296 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:34.420106 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d065cb-ef83-4e56-b3fc-2977c4c0789e None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:34.420948 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d065cb-ef83-4e56-b3fc-2977c4c0789e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:34.422668 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d065cb-ef83-4e56-b3fc-2977c4c0789e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:34.439356 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-fb87120e-5c9f-4985-b7e1-e81294e80fcf None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:34.441843 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-fb87120e-5c9f-4985-b7e1-e81294e80fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:34.454069 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-f1d065cb-ef83-4e56-b3fc-2977c4c0789e None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:34.454069 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f1d065cb-ef83-4e56-b3fc-2977c4c0789e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:34.458129 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 776/1546] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:34.459549 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782342bf-fbf5-46d9-aa81-28bc72aa98eb None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:34.459842 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782342bf-fbf5-46d9-aa81-28bc72aa98eb None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:34.460126 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782342bf-fbf5-46d9-aa81-28bc72aa98eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:34.464149 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 771/1547] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:34.466220 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d065cb-ef83-4e56-b3fc-2977c4c0789e None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:34.476808 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d065cb-ef83-4e56-b3fc-2977c4c0789e None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:34.483013 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58372851-cacc-4cbf-82ab-9d9facc44740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:34.483642 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58372851-cacc-4cbf-82ab-9d9facc44740 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:34.483928 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58372851-cacc-4cbf-82ab-9d9facc44740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:34.484202 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58372851-cacc-4cbf-82ab-9d9facc44740 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:34.484903 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58372851-cacc-4cbf-82ab-9d9facc44740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:34.491701 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58372851-cacc-4cbf-82ab-9d9facc44740 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:34.492469 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-782342bf-fbf5-46d9-aa81-28bc72aa98eb None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:34.493699 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-782342bf-fbf5-46d9-aa81-28bc72aa98eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:34.502661 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 777/1548] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:34.513460 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 772/1549] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:34.513591 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139cf755-c264-4974-8014-9381d467e06d None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:34.513687 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed99ba4-6150-4b02-b558-68302444ea22 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:34.513791 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139cf755-c264-4974-8014-9381d467e06d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:34.513901 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed99ba4-6150-4b02-b558-68302444ea22 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:34.514026 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139cf755-c264-4974-8014-9381d467e06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:34.514118 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed99ba4-6150-4b02-b558-68302444ea22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:34.524739 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-139cf755-c264-4974-8014-9381d467e06d None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:34.526533 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-139cf755-c264-4974-8014-9381d467e06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:34.531795 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-2ed99ba4-6150-4b02-b558-68302444ea22 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:34.533017 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2ed99ba4-6150-4b02-b558-68302444ea22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:34.534566 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 778/1550] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:34.535759 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f543ae-4576-40aa-94e9-28464dcf46de None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:34.536048 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f543ae-4576-40aa-94e9-28464dcf46de None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:34.536323 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f543ae-4576-40aa-94e9-28464dcf46de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:34.545186 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 773/1551] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:34.546464 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc9b92-db63-4401-91ae-7370fcc12277 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:34.546748 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc9b92-db63-4401-91ae-7370fcc12277 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:34.547021 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc9b92-db63-4401-91ae-7370fcc12277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:34.561078 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-43f543ae-4576-40aa-94e9-28464dcf46de None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:34.562326 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-43f543ae-4576-40aa-94e9-28464dcf46de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:34.566161 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-32dc9b92-db63-4401-91ae-7370fcc12277 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:34.567309 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-32dc9b92-db63-4401-91ae-7370fcc12277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:34.570513 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 779/1552] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:34.571667 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bae67e-138b-4be7-911a-46381cf1f769 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:34.571952 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bae67e-138b-4be7-911a-46381cf1f769 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:34.572232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bae67e-138b-4be7-911a-46381cf1f769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:34.581178 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 774/1553] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:34.582691 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba4fa9b-0103-40fa-a801-179ba8fa4e12 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:34.582976 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba4fa9b-0103-40fa-a801-179ba8fa4e12 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:34.583249 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba4fa9b-0103-40fa-a801-179ba8fa4e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:34.594491 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-f2bae67e-138b-4be7-911a-46381cf1f769 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:34.595698 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f2bae67e-138b-4be7-911a-46381cf1f769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:34.602476 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-eba4fa9b-0103-40fa-a801-179ba8fa4e12 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:34.603617 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-eba4fa9b-0103-40fa-a801-179ba8fa4e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:34.604848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 780/1554] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:34.608745 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a01770-d870-4308-b0d1-167c7a9a3c3a None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:34.608745 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a01770-d870-4308-b0d1-167c7a9a3c3a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:34.608745 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a01770-d870-4308-b0d1-167c7a9a3c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:34.614173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 775/1555] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:34.614922 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba4fa9b-0103-40fa-a801-179ba8fa4e12 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:34.624398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba4fa9b-0103-40fa-a801-179ba8fa4e12 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:34.628857 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b768c7b9-86e1-4d9d-bec7-f5fad3a0b308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:34.628996 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-26a01770-d870-4308-b0d1-167c7a9a3c3a None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:34.629481 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768c7b9-86e1-4d9d-bec7-f5fad3a0b308 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:34.629761 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768c7b9-86e1-4d9d-bec7-f5fad3a0b308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:34.630056 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768c7b9-86e1-4d9d-bec7-f5fad3a0b308 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:34.630174 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-26a01770-d870-4308-b0d1-167c7a9a3c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:34.630749 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b768c7b9-86e1-4d9d-bec7-f5fad3a0b308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:34.632337 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b768c7b9-86e1-4d9d-bec7-f5fad3a0b308 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:34.638896 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 781/1556] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:34.639722 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a01770-d870-4308-b0d1-167c7a9a3c3a None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:34.643285 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 776/1557] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:34.644494 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bc064-ecff-4e2a-b844-9e2f156a1117 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:34.644781 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bc064-ecff-4e2a-b844-9e2f156a1117 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:34.645061 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bc064-ecff-4e2a-b844-9e2f156a1117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:34.649030 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a01770-d870-4308-b0d1-167c7a9a3c3a None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:34.654439 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6745d90-ae97-4755-83bc-7c3dc9c1f6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:34.655054 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6745d90-ae97-4755-83bc-7c3dc9c1f6de admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:34.655335 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6745d90-ae97-4755-83bc-7c3dc9c1f6de admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:34.655612 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6745d90-ae97-4755-83bc-7c3dc9c1f6de admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:34.656339 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6745d90-ae97-4755-83bc-7c3dc9c1f6de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:21:34.656643 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6745d90-ae97-4755-83bc-7c3dc9c1f6de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:34.658317 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6745d90-ae97-4755-83bc-7c3dc9c1f6de admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:34.664930 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-bb4bc064-ecff-4e2a-b844-9e2f156a1117 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:34.666196 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-bb4bc064-ecff-4e2a-b844-9e2f156a1117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:34.670783 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 782/1558] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:21:34.671975 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcb725-eebd-48f0-b38e-909543a371a5 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:34.672260 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcb725-eebd-48f0-b38e-909543a371a5 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:34.672536 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcb725-eebd-48f0-b38e-909543a371a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:34.682051 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 777/1559] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:34.683053 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4bc064-ecff-4e2a-b844-9e2f156a1117 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:34.691887 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-7ebcb725-eebd-48f0-b38e-909543a371a5 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:34.692590 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4bc064-ecff-4e2a-b844-9e2f156a1117 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:34.693073 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7ebcb725-eebd-48f0-b38e-909543a371a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:34.697512 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247f4baa-e2b1-4c82-b9bc-9c9081f29649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:34.698186 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f4baa-e2b1-4c82-b9bc-9c9081f29649 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:34.698478 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f4baa-e2b1-4c82-b9bc-9c9081f29649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:34.698756 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f4baa-e2b1-4c82-b9bc-9c9081f29649 admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:34.699453 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247f4baa-e2b1-4c82-b9bc-9c9081f29649 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:21:34.699757 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247f4baa-e2b1-4c82-b9bc-9c9081f29649 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:34.700769 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: WARNING py.warnings [None req-247f4baa-e2b1-4c82-b9bc-9c9081f29649 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 27 04:21:34.700769 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: warnings.warn(msg) Jul 27 04:21:34.700769 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: Jul 27 04:21:34.701561 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 783/1560] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:34.701665 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247f4baa-e2b1-4c82-b9bc-9c9081f29649 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:34.702357 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebcb725-eebd-48f0-b38e-909543a371a5 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:34.711186 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebcb725-eebd-48f0-b38e-909543a371a5 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:34.717121 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 778/1561] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:21:34.717121 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467325e1-b8c2-4b43-baa5-6cc7966879b8 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:34.717121 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467325e1-b8c2-4b43-baa5-6cc7966879b8 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:34.717121 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467325e1-b8c2-4b43-baa5-6cc7966879b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:34.719300 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9406db-2d4b-47bd-9593-e781eda405f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:34.719914 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9406db-2d4b-47bd-9593-e781eda405f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:34.720193 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9406db-2d4b-47bd-9593-e781eda405f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:34.720462 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9406db-2d4b-47bd-9593-e781eda405f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:34.721225 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9406db-2d4b-47bd-9593-e781eda405f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:21:34.721534 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9406db-2d4b-47bd-9593-e781eda405f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:34.723137 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9406db-2d4b-47bd-9593-e781eda405f6 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:34.738761 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 784/1562] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:21:34.738761 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9406db-2d4b-47bd-9593-e781eda405f6 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:34.739010 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-467325e1-b8c2-4b43-baa5-6cc7966879b8 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:34.739010 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-467325e1-b8c2-4b43-baa5-6cc7966879b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:34.745809 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 779/1563] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:34.749388 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467325e1-b8c2-4b43-baa5-6cc7966879b8 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:34.749574 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9406db-2d4b-47bd-9593-e781eda405f6 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:34.752443 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070322b-f4f0-4fc2-a9be-5128a3829a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:34.753046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070322b-f4f0-4fc2-a9be-5128a3829a67 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:34.753326 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070322b-f4f0-4fc2-a9be-5128a3829a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:34.753598 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070322b-f4f0-4fc2-a9be-5128a3829a67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:34.754321 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070322b-f4f0-4fc2-a9be-5128a3829a67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:21:34.754649 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070322b-f4f0-4fc2-a9be-5128a3829a67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:34.755928 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467325e1-b8c2-4b43-baa5-6cc7966879b8 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:34.756229 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070322b-f4f0-4fc2-a9be-5128a3829a67 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:34.760556 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498ffbbd-d715-4915-bb67-c0ab4d3216c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:34.761314 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ffbbd-d715-4915-bb67-c0ab4d3216c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:34.761599 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ffbbd-d715-4915-bb67-c0ab4d3216c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:34.761884 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ffbbd-d715-4915-bb67-c0ab4d3216c5 admin admin] PATH_INFO: `/v3/projects/5bb7f3d2bc1e4651b7f1e0ac6debd89a/users/5a69202d87094049b12765d64c17e515/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:34.771489 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 785/1564] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:21:34.772505 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070322b-f4f0-4fc2-a9be-5128a3829a67 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:34.777878 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498ffbbd-d715-4915-bb67-c0ab4d3216c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bb7f3d2bc1e4651b7f1e0ac6debd89a, user_id=5a69202d87094049b12765d64c17e515, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:34.780916 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498ffbbd-d715-4915-bb67-c0ab4d3216c5 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:34.784801 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070322b-f4f0-4fc2-a9be-5128a3829a67 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:34.789745 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e208c-2e51-4ce8-b278-b5f429db37cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:34.790413 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e208c-2e51-4ce8-b278-b5f429db37cd admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:34.790692 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e208c-2e51-4ce8-b278-b5f429db37cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:34.790972 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e208c-2e51-4ce8-b278-b5f429db37cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:34.792218 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8e208c-2e51-4ce8-b278-b5f429db37cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:34.793227 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: WARNING py.warnings [None req-2d8e208c-2e51-4ce8-b278-b5f429db37cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 04:21:34.793227 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: warnings.warn(msg) Jul 27 04:21:34.793227 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: Jul 27 04:21:34.794436 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8e208c-2e51-4ce8-b278-b5f429db37cd admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:34.794885 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 780/1565] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:21:34 2022] PUT /identity/v3/projects/5bb7f3d2bc1e4651b7f1e0ac6debd89a/users/5a69202d87094049b12765d64c17e515/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:34.795528 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498ffbbd-d715-4915-bb67-c0ab4d3216c5 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:34.809627 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:34.825379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 786/1566] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:21:34 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:34.825379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e208c-2e51-4ce8-b278-b5f429db37cd admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:34.857926 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:34.861343 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498ffbbd-d715-4915-bb67-c0ab4d3216c5 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:34.887142 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e208c-2e51-4ce8-b278-b5f429db37cd admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:34.892206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2564df-1cbc-43be-970a-b9a1d658da91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:34.892861 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2564df-1cbc-43be-970a-b9a1d658da91 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:34.893141 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2564df-1cbc-43be-970a-b9a1d658da91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:34.893428 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2564df-1cbc-43be-970a-b9a1d658da91 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:34.894153 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2564df-1cbc-43be-970a-b9a1d658da91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:21:34.894458 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2564df-1cbc-43be-970a-b9a1d658da91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:34.896169 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2564df-1cbc-43be-970a-b9a1d658da91 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:34.908078 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 787/1567] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:21:34.908902 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2564df-1cbc-43be-970a-b9a1d658da91 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:34.919076 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2564df-1cbc-43be-970a-b9a1d658da91 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:34.923776 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98617a88-18df-47f0-84b2-080b2e5e6198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:34.924383 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98617a88-18df-47f0-84b2-080b2e5e6198 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:34.924670 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98617a88-18df-47f0-84b2-080b2e5e6198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:34.924952 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98617a88-18df-47f0-84b2-080b2e5e6198 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:34.925658 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98617a88-18df-47f0-84b2-080b2e5e6198 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:34.926491 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8becf086-47f9-45c4-9992-6790b8aaa35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:34.927119 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8becf086-47f9-45c4-9992-6790b8aaa35d admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:34.927400 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8becf086-47f9-45c4-9992-6790b8aaa35d admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:34.927593 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98617a88-18df-47f0-84b2-080b2e5e6198 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:34.927690 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8becf086-47f9-45c4-9992-6790b8aaa35d admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:34.928367 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8becf086-47f9-45c4-9992-6790b8aaa35d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:21:34.928676 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8becf086-47f9-45c4-9992-6790b8aaa35d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:34.930291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8becf086-47f9-45c4-9992-6790b8aaa35d admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:34.946015 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 781/1568] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:21:34.946015 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8becf086-47f9-45c4-9992-6790b8aaa35d admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:34.954779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8becf086-47f9-45c4-9992-6790b8aaa35d admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:34.959930 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93675fe7-4699-4ccd-9e78-5c0ac6a89ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:34.960536 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93675fe7-4699-4ccd-9e78-5c0ac6a89ea9 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:34.960824 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93675fe7-4699-4ccd-9e78-5c0ac6a89ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:34.961103 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93675fe7-4699-4ccd-9e78-5c0ac6a89ea9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:34.961772 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93675fe7-4699-4ccd-9e78-5c0ac6a89ea9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:34.967580 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 788/1569] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:21:34 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:34.967742 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93675fe7-4699-4ccd-9e78-5c0ac6a89ea9 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:34.967852 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98617a88-18df-47f0-84b2-080b2e5e6198 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:34.996637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 782/1570] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:21:34 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:34.996637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93675fe7-4699-4ccd-9e78-5c0ac6a89ea9 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:35.029098 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98617a88-18df-47f0-84b2-080b2e5e6198 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:35.056764 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93675fe7-4699-4ccd-9e78-5c0ac6a89ea9 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:35.061878 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a470a3-76f8-4b9f-9b76-5e65356b669c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:35.062537 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a470a3-76f8-4b9f-9b76-5e65356b669c admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:35.062815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a470a3-76f8-4b9f-9b76-5e65356b669c admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:35.063093 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a470a3-76f8-4b9f-9b76-5e65356b669c admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:35.063779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a470a3-76f8-4b9f-9b76-5e65356b669c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:35.065674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a470a3-76f8-4b9f-9b76-5e65356b669c admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:35.085403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9ca50-34c2-4258-8658-f1777da85389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:35.086057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9ca50-34c2-4258-8658-f1777da85389 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:35.086419 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9ca50-34c2-4258-8658-f1777da85389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:35.086699 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9ca50-34c2-4258-8658-f1777da85389 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:35.087403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f9ca50-34c2-4258-8658-f1777da85389 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:21:35.087704 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f9ca50-34c2-4258-8658-f1777da85389 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:35.089546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f9ca50-34c2-4258-8658-f1777da85389 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:35.092978 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 783/1571] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:21:34 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:35.094258 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a384d9-8f94-47dd-ac01-6f8fab3452a3 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:35.094549 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a384d9-8f94-47dd-ac01-6f8fab3452a3 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:35.094830 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a384d9-8f94-47dd-ac01-6f8fab3452a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:35.101470 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 789/1572] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:21:35.102431 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9ca50-34c2-4258-8658-f1777da85389 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:35.113065 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9ca50-34c2-4258-8658-f1777da85389 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:35.121453 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-a3a384d9-8f94-47dd-ac01-6f8fab3452a3 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:35.122700 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-a3a384d9-8f94-47dd-ac01-6f8fab3452a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:35.136180 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:35.157954 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:35.171946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc0fd4-374b-4a03-bdfd-812458e64769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:35.172510 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc0fd4-374b-4a03-bdfd-812458e64769 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:35.172792 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc0fd4-374b-4a03-bdfd-812458e64769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:35.173073 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc0fd4-374b-4a03-bdfd-812458e64769 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:35.173745 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc0fd4-374b-4a03-bdfd-812458e64769 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:35.175714 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc0fd4-374b-4a03-bdfd-812458e64769 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:35.185181 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 784/1573] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:35.188428 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a384d9-8f94-47dd-ac01-6f8fab3452a3 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:35.200104 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a384d9-8f94-47dd-ac01-6f8fab3452a3 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:35.203536 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 790/1574] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:21:35 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:35.203536 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc0fd4-374b-4a03-bdfd-812458e64769 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:35.205099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644abf3-50a7-4c8d-8ce2-81f3170500bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:35.205725 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644abf3-50a7-4c8d-8ce2-81f3170500bd admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:35.206040 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644abf3-50a7-4c8d-8ce2-81f3170500bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:35.206350 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644abf3-50a7-4c8d-8ce2-81f3170500bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:35.207024 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d644abf3-50a7-4c8d-8ce2-81f3170500bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:35.208946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d644abf3-50a7-4c8d-8ce2-81f3170500bd admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:35.211347 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc0fd4-374b-4a03-bdfd-812458e64769 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:35.216261 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d95a01-0c9e-4f0d-961e-8e27ea59980a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:35.216928 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d95a01-0c9e-4f0d-961e-8e27ea59980a admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:35.217216 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d95a01-0c9e-4f0d-961e-8e27ea59980a admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:35.217547 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d95a01-0c9e-4f0d-961e-8e27ea59980a admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:35.218256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d95a01-0c9e-4f0d-961e-8e27ea59980a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:35.220188 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d95a01-0c9e-4f0d-961e-8e27ea59980a admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:35.238190 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 785/1575] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:21:35 2022] POST /identity/v3/projects => generated 405 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:35.242006 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644abf3-50a7-4c8d-8ce2-81f3170500bd admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:35.255248 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 791/1576] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:21:35 2022] POST /identity/v3/projects => generated 403 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:35.255990 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d95a01-0c9e-4f0d-961e-8e27ea59980a admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:35.304598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644abf3-50a7-4c8d-8ce2-81f3170500bd admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:35.322757 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d95a01-0c9e-4f0d-961e-8e27ea59980a admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:35.327528 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5076e-e27b-40b6-847d-305ddd9666df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:35.328151 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5076e-e27b-40b6-847d-305ddd9666df admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:35.328436 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5076e-e27b-40b6-847d-305ddd9666df admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:35.328716 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5076e-e27b-40b6-847d-305ddd9666df admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:35.329383 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff5076e-e27b-40b6-847d-305ddd9666df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:35.331390 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff5076e-e27b-40b6-847d-305ddd9666df admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:35.353408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 792/1577] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:21:35 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:35.355020 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5076e-e27b-40b6-847d-305ddd9666df admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:35.363834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96446650-0cd2-4f99-ba7d-355bfbf97b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:35.364468 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96446650-0cd2-4f99-ba7d-355bfbf97b0d admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:35.364595 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5076e-e27b-40b6-847d-305ddd9666df admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:35.364892 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96446650-0cd2-4f99-ba7d-355bfbf97b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:35.365197 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96446650-0cd2-4f99-ba7d-355bfbf97b0d admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:35.365860 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96446650-0cd2-4f99-ba7d-355bfbf97b0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:35.367823 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96446650-0cd2-4f99-ba7d-355bfbf97b0d admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:35.369334 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899e18f-d5df-48ef-b3b0-977b2c782032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:35.370943 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899e18f-d5df-48ef-b3b0-977b2c782032 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:35.371240 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899e18f-d5df-48ef-b3b0-977b2c782032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:35.371523 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899e18f-d5df-48ef-b3b0-977b2c782032 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:35.372438 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c899e18f-d5df-48ef-b3b0-977b2c782032 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:35.376731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c899e18f-d5df-48ef-b3b0-977b2c782032 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:35.393804 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 786/1578] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:21:35 2022] POST /identity/v3/users => generated 290 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:35.394954 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96446650-0cd2-4f99-ba7d-355bfbf97b0d admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:35.401012 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 793/1579] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:21:35 2022] POST /identity/v3/users => generated 287 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:35.402131 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899e18f-d5df-48ef-b3b0-977b2c782032 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:35.407993 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96446650-0cd2-4f99-ba7d-355bfbf97b0d admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:35.412623 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8013e5-011d-4aec-9ba8-0b19d9910b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:35.413244 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8013e5-011d-4aec-9ba8-0b19d9910b17 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:35.413529 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8013e5-011d-4aec-9ba8-0b19d9910b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:35.413803 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8013e5-011d-4aec-9ba8-0b19d9910b17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:35.414509 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8013e5-011d-4aec-9ba8-0b19d9910b17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:35.416505 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8013e5-011d-4aec-9ba8-0b19d9910b17 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:35.445149 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 787/1580] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:21:35 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:35.446919 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8013e5-011d-4aec-9ba8-0b19d9910b17 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:35.462081 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899e18f-d5df-48ef-b3b0-977b2c782032 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:35.472008 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:35.504853 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:35.515784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7418a4-f6ac-48fb-ab62-0ecc477957df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:35.516418 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7418a4-f6ac-48fb-ab62-0ecc477957df service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:35.516701 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7418a4-f6ac-48fb-ab62-0ecc477957df service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:35.516983 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7418a4-f6ac-48fb-ab62-0ecc477957df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:35.522666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8013e5-011d-4aec-9ba8-0b19d9910b17 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:35.526180 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7418a4-f6ac-48fb-ab62-0ecc477957df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:35.527588 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440df8a-bdc5-46cd-ae1e-5e7f4dc20cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:35.527734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7418a4-f6ac-48fb-ab62-0ecc477957df service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:35.528209 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440df8a-bdc5-46cd-ae1e-5e7f4dc20cc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:35.528496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440df8a-bdc5-46cd-ae1e-5e7f4dc20cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:35.528774 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440df8a-bdc5-46cd-ae1e-5e7f4dc20cc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:35.529477 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440df8a-bdc5-46cd-ae1e-5e7f4dc20cc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:35.531113 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440df8a-bdc5-46cd-ae1e-5e7f4dc20cc4 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:35.537156 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 794/1581] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:21:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:21:35.537925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7418a4-f6ac-48fb-ab62-0ecc477957df service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:35.542723 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 788/1582] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:35.543909 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440df8a-bdc5-46cd-ae1e-5e7f4dc20cc4 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:35.549488 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7418a4-f6ac-48fb-ab62-0ecc477957df service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:35.554430 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd66b31-a689-4ac7-a237-8fd1cb6b8119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:35.555075 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd66b31-a689-4ac7-a237-8fd1cb6b8119 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:35.555355 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd66b31-a689-4ac7-a237-8fd1cb6b8119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:35.555556 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440df8a-bdc5-46cd-ae1e-5e7f4dc20cc4 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:35.555687 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd66b31-a689-4ac7-a237-8fd1cb6b8119 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:35.556688 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd66b31-a689-4ac7-a237-8fd1cb6b8119 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:35.559106 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd66b31-a689-4ac7-a237-8fd1cb6b8119 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:35.561400 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf99e2c-b7b8-415d-94a0-5b7fe6df01fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:35.562091 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf99e2c-b7b8-415d-94a0-5b7fe6df01fc admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:35.562597 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf99e2c-b7b8-415d-94a0-5b7fe6df01fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:35.562894 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf99e2c-b7b8-415d-94a0-5b7fe6df01fc admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:35.563746 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf99e2c-b7b8-415d-94a0-5b7fe6df01fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:35.566271 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf99e2c-b7b8-415d-94a0-5b7fe6df01fc admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:35.587350 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 795/1583] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:21:35 2022] POST /identity/v3/users => generated 303 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:35.587350 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd66b31-a689-4ac7-a237-8fd1cb6b8119 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:35.596902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 789/1584] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:21:35 2022] POST /identity/v3/users => generated 302 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:35.598292 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd66b31-a689-4ac7-a237-8fd1cb6b8119 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:35.598441 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf99e2c-b7b8-415d-94a0-5b7fe6df01fc admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:35.603205 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf94d9-784a-40a1-8223-e5d5c3d9d827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:35.604003 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf94d9-784a-40a1-8223-e5d5c3d9d827 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:35.604293 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf94d9-784a-40a1-8223-e5d5c3d9d827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:35.604582 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf94d9-784a-40a1-8223-e5d5c3d9d827 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:35.605383 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bf94d9-784a-40a1-8223-e5d5c3d9d827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:35.607243 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bf94d9-784a-40a1-8223-e5d5c3d9d827 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:35.608692 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf99e2c-b7b8-415d-94a0-5b7fe6df01fc admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:35.614339 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f149d-853d-4e77-a451-e0725c9e1777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:35.615272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f149d-853d-4e77-a451-e0725c9e1777 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:35.615424 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f149d-853d-4e77-a451-e0725c9e1777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:35.615600 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f149d-853d-4e77-a451-e0725c9e1777 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:35.616326 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f149d-853d-4e77-a451-e0725c9e1777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:35.618033 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f149d-853d-4e77-a451-e0725c9e1777 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:35.619503 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 796/1585] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:35.620858 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf94d9-784a-40a1-8223-e5d5c3d9d827 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:35.633747 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 790/1586] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:35.633747 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f149d-853d-4e77-a451-e0725c9e1777 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:35.634327 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf94d9-784a-40a1-8223-e5d5c3d9d827 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:35.639496 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f9853-4e84-4e3d-af6f-4a6407277fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:35.640132 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f9853-4e84-4e3d-af6f-4a6407277fe8 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:35.640444 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f9853-4e84-4e3d-af6f-4a6407277fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:35.640696 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f9853-4e84-4e3d-af6f-4a6407277fe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:35.641562 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0f9853-4e84-4e3d-af6f-4a6407277fe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:35.643057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0f9853-4e84-4e3d-af6f-4a6407277fe8 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:35.646162 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f149d-853d-4e77-a451-e0725c9e1777 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:35.652498 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc10d18-4c1b-4395-be72-0c0bf4653cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:35.653178 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc10d18-4c1b-4395-be72-0c0bf4653cc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:35.653543 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc10d18-4c1b-4395-be72-0c0bf4653cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:35.653863 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc10d18-4c1b-4395-be72-0c0bf4653cc1 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:35.654713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc10d18-4c1b-4395-be72-0c0bf4653cc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:35.657141 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc10d18-4c1b-4395-be72-0c0bf4653cc1 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:35.657283 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 797/1587] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:35.658135 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f9853-4e84-4e3d-af6f-4a6407277fe8 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:35.668660 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f9853-4e84-4e3d-af6f-4a6407277fe8 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:35.674166 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d237db8-d33e-46dc-b7fc-a939bf8abdc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:35.674847 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d237db8-d33e-46dc-b7fc-a939bf8abdc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:35.675134 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d237db8-d33e-46dc-b7fc-a939bf8abdc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:35.675400 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d237db8-d33e-46dc-b7fc-a939bf8abdc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:35.676104 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d237db8-d33e-46dc-b7fc-a939bf8abdc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:35.677854 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d237db8-d33e-46dc-b7fc-a939bf8abdc9 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:35.686034 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 791/1588] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:21:35 2022] POST /identity/v3/users => generated 284 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:35.686034 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc10d18-4c1b-4395-be72-0c0bf4653cc1 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:35.691925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 798/1589] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:35.692789 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d237db8-d33e-46dc-b7fc-a939bf8abdc9 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:35.698952 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc10d18-4c1b-4395-be72-0c0bf4653cc1 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:35.704117 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289fc9d-3fce-4847-a778-7e5c30171e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:35.704301 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d237db8-d33e-46dc-b7fc-a939bf8abdc9 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:35.704742 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289fc9d-3fce-4847-a778-7e5c30171e56 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:35.705064 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289fc9d-3fce-4847-a778-7e5c30171e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:35.705305 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289fc9d-3fce-4847-a778-7e5c30171e56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:35.706058 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a289fc9d-3fce-4847-a778-7e5c30171e56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:35.707649 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a289fc9d-3fce-4847-a778-7e5c30171e56 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:35.709438 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634660ff-158f-4438-97f4-739b175938f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:35.710199 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634660ff-158f-4438-97f4-739b175938f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:35.710611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634660ff-158f-4438-97f4-739b175938f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:35.710989 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634660ff-158f-4438-97f4-739b175938f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:35.711803 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634660ff-158f-4438-97f4-739b175938f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:35.714225 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634660ff-158f-4438-97f4-739b175938f9 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:35.726172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 792/1590] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:35.726172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289fc9d-3fce-4847-a778-7e5c30171e56 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:35.732007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 799/1591] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:35.732007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634660ff-158f-4438-97f4-739b175938f9 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:35.736523 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289fc9d-3fce-4847-a778-7e5c30171e56 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:35.741752 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f2ef3-f5d2-4afd-97f8-b911d48d9296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:35.742515 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634660ff-158f-4438-97f4-739b175938f9 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:35.742842 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f2ef3-f5d2-4afd-97f8-b911d48d9296 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:35.743295 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f2ef3-f5d2-4afd-97f8-b911d48d9296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:35.743835 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f2ef3-f5d2-4afd-97f8-b911d48d9296 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:35.745294 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f2ef3-f5d2-4afd-97f8-b911d48d9296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:35.747430 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f2ef3-f5d2-4afd-97f8-b911d48d9296 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:35.748563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd940d9-02fc-473d-a2c9-c18bb6a2ab79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:35.749219 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd940d9-02fc-473d-a2c9-c18bb6a2ab79 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:35.749583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd940d9-02fc-473d-a2c9-c18bb6a2ab79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:35.749960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd940d9-02fc-473d-a2c9-c18bb6a2ab79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:35.750652 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd940d9-02fc-473d-a2c9-c18bb6a2ab79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:35.752558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd940d9-02fc-473d-a2c9-c18bb6a2ab79 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:35.766721 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 793/1592] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:35.766721 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f2ef3-f5d2-4afd-97f8-b911d48d9296 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:35.776740 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f2ef3-f5d2-4afd-97f8-b911d48d9296 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:35.778956 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 800/1593] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:35.779979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd940d9-02fc-473d-a2c9-c18bb6a2ab79 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:35.784974 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be485e1b-50dc-4daf-8094-929bbd055bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:35.785611 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be485e1b-50dc-4daf-8094-929bbd055bc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:35.785897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be485e1b-50dc-4daf-8094-929bbd055bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:35.786216 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be485e1b-50dc-4daf-8094-929bbd055bc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:35.787047 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be485e1b-50dc-4daf-8094-929bbd055bc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:35.788766 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be485e1b-50dc-4daf-8094-929bbd055bc1 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:35.792434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd940d9-02fc-473d-a2c9-c18bb6a2ab79 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:35.797692 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92163039-470b-4515-b0fb-a73e5161f43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:35.798493 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92163039-470b-4515-b0fb-a73e5161f43e admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:35.798777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92163039-470b-4515-b0fb-a73e5161f43e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:35.799088 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92163039-470b-4515-b0fb-a73e5161f43e admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:35.799800 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92163039-470b-4515-b0fb-a73e5161f43e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:35.801690 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92163039-470b-4515-b0fb-a73e5161f43e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:35.804806 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 794/1594] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:35.806177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be485e1b-50dc-4daf-8094-929bbd055bc1 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:35.815031 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 801/1595] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:35.816661 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92163039-470b-4515-b0fb-a73e5161f43e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:35.821055 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be485e1b-50dc-4daf-8094-929bbd055bc1 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:35.826273 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9cd168-3d08-4bd3-8537-db6f7fca69b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:35.827085 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cd168-3d08-4bd3-8537-db6f7fca69b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:35.827463 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cd168-3d08-4bd3-8537-db6f7fca69b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:35.827840 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cd168-3d08-4bd3-8537-db6f7fca69b4 admin admin] PATH_INFO: `/v3/projects/1bd488a9ff8f4e53a3d839ada563f5c1/users/aa6ec6714d3f4836a603fad8174b1b62/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:35.828487 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92163039-470b-4515-b0fb-a73e5161f43e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:35.835151 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa9d98-24b2-4467-b661-96bcf6c16e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:35.835780 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa9d98-24b2-4467-b661-96bcf6c16e22 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:35.836064 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa9d98-24b2-4467-b661-96bcf6c16e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:35.836340 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa9d98-24b2-4467-b661-96bcf6c16e22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:35.837043 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa9d98-24b2-4467-b661-96bcf6c16e22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:35.839000 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa9d98-24b2-4467-b661-96bcf6c16e22 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:35.848703 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9cd168-3d08-4bd3-8537-db6f7fca69b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bd488a9ff8f4e53a3d839ada563f5c1, user_id=aa6ec6714d3f4836a603fad8174b1b62, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:35.849485 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 802/1596] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:35.850217 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa9d98-24b2-4467-b661-96bcf6c16e22 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:35.855349 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9cd168-3d08-4bd3-8537-db6f7fca69b4 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:35.865245 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa9d98-24b2-4467-b661-96bcf6c16e22 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:35.872241 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 795/1597] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:21:35 2022] PUT /identity/v3/projects/1bd488a9ff8f4e53a3d839ada563f5c1/users/aa6ec6714d3f4836a603fad8174b1b62/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:35.874303 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9cd168-3d08-4bd3-8537-db6f7fca69b4 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:35.935052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e4e42-77dc-41e0-a21b-60e9044a8839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:35.935784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e4e42-77dc-41e0-a21b-60e9044a8839 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:35.936132 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e4e42-77dc-41e0-a21b-60e9044a8839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:35.936470 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e4e42-77dc-41e0-a21b-60e9044a8839 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:35.937245 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e4e42-77dc-41e0-a21b-60e9044a8839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:35.938971 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e4e42-77dc-41e0-a21b-60e9044a8839 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:35.953466 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 803/1598] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:35.953466 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e4e42-77dc-41e0-a21b-60e9044a8839 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:35.953768 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9cd168-3d08-4bd3-8537-db6f7fca69b4 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:35.960490 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327fcf2-8dc6-49d0-90bf-29b56c1fd43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:35.961285 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327fcf2-8dc6-49d0-90bf-29b56c1fd43d admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:35.961680 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327fcf2-8dc6-49d0-90bf-29b56c1fd43d admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:35.962093 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327fcf2-8dc6-49d0-90bf-29b56c1fd43d admin admin] PATH_INFO: `/v3/projects/3a0b9dd0d94644b1bbac5f379c8d021c/users/7845ef787bc5446786f5c7ce9dea9863/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:35.965318 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e4e42-77dc-41e0-a21b-60e9044a8839 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:35.970344 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6369faf-7506-44b3-880b-e98d49b69904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:35.971228 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6369faf-7506-44b3-880b-e98d49b69904 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:35.971710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:35.972518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6369faf-7506-44b3-880b-e98d49b69904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:35.972518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6369faf-7506-44b3-880b-e98d49b69904 admin admin] PATH_INFO: `/v3/projects/7c14d53ffc0d4d90b7cbfbb6b3b5a83b/users/a430d7aa4dcf40e399fcb5d1f4bf3f76/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:35.982389 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327fcf2-8dc6-49d0-90bf-29b56c1fd43d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a0b9dd0d94644b1bbac5f379c8d021c, user_id=7845ef787bc5446786f5c7ce9dea9863, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:35.986060 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327fcf2-8dc6-49d0-90bf-29b56c1fd43d admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:36.001480 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6369faf-7506-44b3-880b-e98d49b69904 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c14d53ffc0d4d90b7cbfbb6b3b5a83b, user_id=a430d7aa4dcf40e399fcb5d1f4bf3f76, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:36.002567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: WARNING py.warnings [None req-e6369faf-7506-44b3-880b-e98d49b69904 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 04:21:36.002567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: warnings.warn(msg) Jul 27 04:21:36.002567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: Jul 27 04:21:36.004866 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6369faf-7506-44b3-880b-e98d49b69904 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:36.008149 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 796/1599] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:21:35 2022] PUT /identity/v3/projects/3a0b9dd0d94644b1bbac5f379c8d021c/users/7845ef787bc5446786f5c7ce9dea9863/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:36.008813 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327fcf2-8dc6-49d0-90bf-29b56c1fd43d admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:36.038393 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 804/1600] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:21:35 2022] PUT /identity/v3/projects/7c14d53ffc0d4d90b7cbfbb6b3b5a83b/users/a430d7aa4dcf40e399fcb5d1f4bf3f76/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:36.040031 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6369faf-7506-44b3-880b-e98d49b69904 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:36.076221 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:36.086302 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327fcf2-8dc6-49d0-90bf-29b56c1fd43d admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:36.134778 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6369faf-7506-44b3-880b-e98d49b69904 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:36.141741 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4597ebc-b6a0-47e7-b802-ccbb3d91f664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:36.142555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4597ebc-b6a0-47e7-b802-ccbb3d91f664 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:36.142973 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4597ebc-b6a0-47e7-b802-ccbb3d91f664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:36.143349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4597ebc-b6a0-47e7-b802-ccbb3d91f664 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:36.144659 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4597ebc-b6a0-47e7-b802-ccbb3d91f664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:36.146552 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4597ebc-b6a0-47e7-b802-ccbb3d91f664 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:36.165762 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 805/1601] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:36 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:36.166351 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d0d7d-522d-4799-ab89-dcc8a7543375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:36.167353 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d0d7d-522d-4799-ab89-dcc8a7543375 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:36.167557 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d0d7d-522d-4799-ab89-dcc8a7543375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:36.167729 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4597ebc-b6a0-47e7-b802-ccbb3d91f664 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:36.167892 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d0d7d-522d-4799-ab89-dcc8a7543375 admin admin] PATH_INFO: `/v3/projects/b97850dff881499c9c9d6a66d43adfb4/users/d2b422350e7d4995ba31ccefa9333dcf/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:36.178602 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4597ebc-b6a0-47e7-b802-ccbb3d91f664 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:36.184536 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7315e8-6beb-4b56-b426-852097eb88d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:36.185406 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7315e8-6beb-4b56-b426-852097eb88d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:36.185608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d0d7d-522d-4799-ab89-dcc8a7543375 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b97850dff881499c9c9d6a66d43adfb4, user_id=d2b422350e7d4995ba31ccefa9333dcf, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:36.186038 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7315e8-6beb-4b56-b426-852097eb88d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:36.186510 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7315e8-6beb-4b56-b426-852097eb88d3 admin admin] PATH_INFO: `/v3/projects/92b1920cfcbe49ea9f9a65d9e1c2f5fd/users/90c723ce0dec415494fe64c7e03b2915/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:36.188566 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d0d7d-522d-4799-ab89-dcc8a7543375 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:36.203342 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7315e8-6beb-4b56-b426-852097eb88d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92b1920cfcbe49ea9f9a65d9e1c2f5fd, user_id=90c723ce0dec415494fe64c7e03b2915, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:36.205217 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 797/1602] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:21:36 2022] PUT /identity/v3/projects/b97850dff881499c9c9d6a66d43adfb4/users/d2b422350e7d4995ba31ccefa9333dcf/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:36.206059 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7315e8-6beb-4b56-b426-852097eb88d3 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:36.207137 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10175b-e175-405f-bc84-5588c9e49901 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:36.207559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10175b-e175-405f-bc84-5588c9e49901 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:36.208056 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10175b-e175-405f-bc84-5588c9e49901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:36.232169 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 806/1603] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:21:36 2022] PUT /identity/v3/projects/92b1920cfcbe49ea9f9a65d9e1c2f5fd/users/90c723ce0dec415494fe64c7e03b2915/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:36.232832 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7315e8-6beb-4b56-b426-852097eb88d3 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:36.242745 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-8b10175b-e175-405f-bc84-5588c9e49901 None None] MFA Rules not processed for user `aa6ec6714d3f4836a603fad8174b1b62`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:36.244170 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-8b10175b-e175-405f-bc84-5588c9e49901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:36.308389 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7315e8-6beb-4b56-b426-852097eb88d3 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:36.313225 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a31866e-acd0-48f6-a72a-9d5651e6898f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:36.313893 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31866e-acd0-48f6-a72a-9d5651e6898f admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:36.314210 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31866e-acd0-48f6-a72a-9d5651e6898f admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:36.314492 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31866e-acd0-48f6-a72a-9d5651e6898f admin admin] PATH_INFO: `/v3/projects/904ec432f16b407ea0dd54a3caf637b4/users/97602bd0c5ce4e11819b634995d93405/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:36.330794 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a31866e-acd0-48f6-a72a-9d5651e6898f admin admin] RBAC: Authorizing `identity:create_grant(project_id=904ec432f16b407ea0dd54a3caf637b4, user_id=97602bd0c5ce4e11819b634995d93405, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:36.333534 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a31866e-acd0-48f6-a72a-9d5651e6898f admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:36.333732 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 798/1604] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:36.335137 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42fc85-82cd-400a-a422-288559d00d28 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:36.335526 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42fc85-82cd-400a-a422-288559d00d28 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:36.335906 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42fc85-82cd-400a-a422-288559d00d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:36.351572 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:36.354658 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 807/1605] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:21:36 2022] PUT /identity/v3/projects/904ec432f16b407ea0dd54a3caf637b4/users/97602bd0c5ce4e11819b634995d93405/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:36.357856 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642b801-86d0-46a9-81d3-c6732bc67280 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:36.358267 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642b801-86d0-46a9-81d3-c6732bc67280 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:36.358699 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642b801-86d0-46a9-81d3-c6732bc67280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:36.372897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-2e42fc85-82cd-400a-a422-288559d00d28 None None] MFA Rules not processed for user `7845ef787bc5446786f5c7ce9dea9863`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:36.374741 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2e42fc85-82cd-400a-a422-288559d00d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:36.381799 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-8642b801-86d0-46a9-81d3-c6732bc67280 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:36.383721 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8642b801-86d0-46a9-81d3-c6732bc67280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:36.459888 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 799/1606] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:36.460798 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e42fc85-82cd-400a-a422-288559d00d28 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:36.479410 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 808/1607] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:36.484226 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190b614-bca1-4040-9722-a8f9bd57668d None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:36.484226 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190b614-bca1-4040-9722-a8f9bd57668d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:36.484226 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190b614-bca1-4040-9722-a8f9bd57668d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:36.525313 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-2190b614-bca1-4040-9722-a8f9bd57668d None None] MFA Rules not processed for user `d2b422350e7d4995ba31ccefa9333dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:36.527584 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2190b614-bca1-4040-9722-a8f9bd57668d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:36.535861 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:36.575926 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e42fc85-82cd-400a-a422-288559d00d28 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:36.582450 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a054d212-15c6-4237-b2f2-9ffe785bb5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:36.583237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a054d212-15c6-4237-b2f2-9ffe785bb5ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:36.583597 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a054d212-15c6-4237-b2f2-9ffe785bb5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:36.583944 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a054d212-15c6-4237-b2f2-9ffe785bb5ec admin admin] PATH_INFO: `/v3/projects/8ed40de01a1f47b39b6c122bec5013c8/users/b582f8afe083473f9f9f1a722d791e89/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:36.605879 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a054d212-15c6-4237-b2f2-9ffe785bb5ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ed40de01a1f47b39b6c122bec5013c8, user_id=b582f8afe083473f9f9f1a722d791e89, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:36.609367 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a054d212-15c6-4237-b2f2-9ffe785bb5ec admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:36.619962 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 809/1608] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:36.619962 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075b63c-35f6-45e6-8cd6-d6642b830afb None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:36.619962 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075b63c-35f6-45e6-8cd6-d6642b830afb None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:36.619962 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075b63c-35f6-45e6-8cd6-d6642b830afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:36.640959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 800/1609] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:21:36 2022] PUT /identity/v3/projects/8ed40de01a1f47b39b6c122bec5013c8/users/b582f8afe083473f9f9f1a722d791e89/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:36.646329 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a054d212-15c6-4237-b2f2-9ffe785bb5ec admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:36.675884 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-e075b63c-35f6-45e6-8cd6-d6642b830afb None None] MFA Rules not processed for user `90c723ce0dec415494fe64c7e03b2915`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:36.678433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e075b63c-35f6-45e6-8cd6-d6642b830afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:36.742920 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a054d212-15c6-4237-b2f2-9ffe785bb5ec admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:36.748360 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc63d8-38f9-4319-b849-4d7b11135cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:36.749051 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc63d8-38f9-4319-b849-4d7b11135cc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:36.749400 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc63d8-38f9-4319-b849-4d7b11135cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:36.749676 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc63d8-38f9-4319-b849-4d7b11135cc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:36.750486 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fc63d8-38f9-4319-b849-4d7b11135cc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:36.752952 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fc63d8-38f9-4319-b849-4d7b11135cc8 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:36.760545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 810/1610] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:36 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:36.760545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5cf69-af7b-4017-8625-1e506f6bd723 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:36.760545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5cf69-af7b-4017-8625-1e506f6bd723 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:36.760545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5cf69-af7b-4017-8625-1e506f6bd723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:36.786989 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-c9f5cf69-af7b-4017-8625-1e506f6bd723 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:36.788589 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c9f5cf69-af7b-4017-8625-1e506f6bd723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:36.788758 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 801/1611] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:21:36 2022] POST /identity/v3/projects => generated 381 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:36.790502 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc63d8-38f9-4319-b849-4d7b11135cc8 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:36.798117 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:36.856661 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc63d8-38f9-4319-b849-4d7b11135cc8 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:36.860850 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5639046-3b1e-45e1-a6ba-77ffdec97d64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:36.861480 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5639046-3b1e-45e1-a6ba-77ffdec97d64 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:36.861755 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5639046-3b1e-45e1-a6ba-77ffdec97d64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:36.862065 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5639046-3b1e-45e1-a6ba-77ffdec97d64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:36.887420 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 811/1612] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:36.888742 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b247e-2538-4108-9e1d-5c8c4d5ecff0 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:36.889035 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b247e-2538-4108-9e1d-5c8c4d5ecff0 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:36.889313 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b247e-2538-4108-9e1d-5c8c4d5ecff0 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:36.891068 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 812/1613] 10.222.0.46 () {54 vars in 962 bytes} [Wed Jul 27 04:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:21:36.896368 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e422d2-90fb-4f02-9c33-a74fa46ce1cb None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:36.896779 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e422d2-90fb-4f02-9c33-a74fa46ce1cb None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:36.896998 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e422d2-90fb-4f02-9c33-a74fa46ce1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:36.929682 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-49e422d2-90fb-4f02-9c33-a74fa46ce1cb None None] MFA Rules not processed for user `b582f8afe083473f9f9f1a722d791e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:36.931085 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-49e422d2-90fb-4f02-9c33-a74fa46ce1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:36.940855 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5639046-3b1e-45e1-a6ba-77ffdec97d64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:36.943220 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5639046-3b1e-45e1-a6ba-77ffdec97d64 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:36.958662 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 802/1614] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:21:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:21:36.964263 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:36.966986 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5238699-f58f-461a-81c9-fabb11a7e3c4 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:36.968397 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5238699-f58f-461a-81c9-fabb11a7e3c4 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:36.969706 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5238699-f58f-461a-81c9-fabb11a7e3c4 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:36.974157 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 803/1615] 10.222.0.46 () {54 vars in 962 bytes} [Wed Jul 27 04:21:36 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:21:36.976055 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5238699-f58f-461a-81c9-fabb11a7e3c4 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:36.997085 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5238699-f58f-461a-81c9-fabb11a7e3c4 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:37.002595 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7fd6e4-4b41-44c1-86fb-3edb8e207caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:37.003487 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7fd6e4-4b41-44c1-86fb-3edb8e207caf admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:37.003854 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7fd6e4-4b41-44c1-86fb-3edb8e207caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:37.004245 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7fd6e4-4b41-44c1-86fb-3edb8e207caf admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:37.005108 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7fd6e4-4b41-44c1-86fb-3edb8e207caf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:37.007363 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7fd6e4-4b41-44c1-86fb-3edb8e207caf admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:37.037559 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 813/1616] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:36 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:37.038651 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e422d2-90fb-4f02-9c33-a74fa46ce1cb None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:37.056412 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e422d2-90fb-4f02-9c33-a74fa46ce1cb None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:37.063550 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4865670-1582-498c-9e5f-e3e6b74df825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:37.064651 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4865670-1582-498c-9e5f-e3e6b74df825 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:37.065086 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4865670-1582-498c-9e5f-e3e6b74df825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:37.065418 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 804/1617] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:21:36 2022] POST /identity/v3/projects => generated 403 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:37.068801 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6b51d-0262-4833-acf9-ee93502df728 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:37.069468 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4865670-1582-498c-9e5f-e3e6b74df825 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:37.070343 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4865670-1582-498c-9e5f-e3e6b74df825 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:37.070468 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6b51d-0262-4833-acf9-ee93502df728 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:37.070888 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6b51d-0262-4833-acf9-ee93502df728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:37.073194 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4865670-1582-498c-9e5f-e3e6b74df825 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:37.105374 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 814/1618] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:21:37 2022] POST /identity/v3/users => generated 297 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:37.106833 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4865670-1582-498c-9e5f-e3e6b74df825 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:37.123085 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4865670-1582-498c-9e5f-e3e6b74df825 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:37.134576 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-dfa6b51d-0262-4833-acf9-ee93502df728 None None] MFA Rules not processed for user `9dc9ea8f89c14a2a8e3272f750647dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:37.136676 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-dfa6b51d-0262-4833-acf9-ee93502df728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:37.209014 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2692e420-deb5-4419-b20d-cce1ecf4b414 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:37.209655 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692e420-deb5-4419-b20d-cce1ecf4b414 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:37.209936 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692e420-deb5-4419-b20d-cce1ecf4b414 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:37.210245 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692e420-deb5-4419-b20d-cce1ecf4b414 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:37.233942 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 805/1619] 10.222.0.46 () {58 vars in 1059 bytes} [Wed Jul 27 04:21:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:37.235650 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27cf87-5ee4-46b5-8505-af4ff2f6df43 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:37.236065 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27cf87-5ee4-46b5-8505-af4ff2f6df43 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:37.236441 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27cf87-5ee4-46b5-8505-af4ff2f6df43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:37.265109 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-bd27cf87-5ee4-46b5-8505-af4ff2f6df43 None None] MFA Rules not processed for user `9dc9ea8f89c14a2a8e3272f750647dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:37.266935 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-bd27cf87-5ee4-46b5-8505-af4ff2f6df43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:37.286371 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 806/1620] 10.222.0.46 () {58 vars in 1059 bytes} [Wed Jul 27 04:21:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:37.288486 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd27cf87-5ee4-46b5-8505-af4ff2f6df43 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:37.300346 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:37.340298 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2692e420-deb5-4419-b20d-cce1ecf4b414 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:37.343480 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2692e420-deb5-4419-b20d-cce1ecf4b414 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:37.366162 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 815/1621] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:21:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:21:37.366863 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2692e420-deb5-4419-b20d-cce1ecf4b414 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:37.378012 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd27cf87-5ee4-46b5-8505-af4ff2f6df43 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:37.382982 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2692e420-deb5-4419-b20d-cce1ecf4b414 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:37.394359 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba554c7-63a0-4e40-96ab-50b1da77968f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:37.395552 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba554c7-63a0-4e40-96ab-50b1da77968f admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:37.396525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba554c7-63a0-4e40-96ab-50b1da77968f admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:37.397114 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba554c7-63a0-4e40-96ab-50b1da77968f admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:37.398461 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba554c7-63a0-4e40-96ab-50b1da77968f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:37.399255 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21ca5f-3375-4b72-bab3-32de517d5f6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:37.402117 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21ca5f-3375-4b72-bab3-32de517d5f6a service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:37.402687 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21ca5f-3375-4b72-bab3-32de517d5f6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:37.403096 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21ca5f-3375-4b72-bab3-32de517d5f6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:37.403749 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba554c7-63a0-4e40-96ab-50b1da77968f admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:37.454201 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 816/1622] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:21:37 2022] POST /identity/v3/users => generated 308 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:37.454201 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba554c7-63a0-4e40-96ab-50b1da77968f admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:37.469343 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba554c7-63a0-4e40-96ab-50b1da77968f admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:37.477233 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfde9d5-8c48-491b-9088-3d66e5d44bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:37.478062 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfde9d5-8c48-491b-9088-3d66e5d44bf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:37.478461 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfde9d5-8c48-491b-9088-3d66e5d44bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:37.478844 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfde9d5-8c48-491b-9088-3d66e5d44bf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:37.480199 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfde9d5-8c48-491b-9088-3d66e5d44bf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:37.481938 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfde9d5-8c48-491b-9088-3d66e5d44bf4 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:37.518662 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 817/1623] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:37 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:37.519651 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfde9d5-8c48-491b-9088-3d66e5d44bf4 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:37.571597 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfde9d5-8c48-491b-9088-3d66e5d44bf4 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:37.575876 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa21ca5f-3375-4b72-bab3-32de517d5f6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:37.589468 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa21ca5f-3375-4b72-bab3-32de517d5f6a service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:37.595386 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e95af5-746e-4d2f-a16b-73c012331149 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:37.596165 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e95af5-746e-4d2f-a16b-73c012331149 service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:37.600822 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e95af5-746e-4d2f-a16b-73c012331149 service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:37.603185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e95af5-746e-4d2f-a16b-73c012331149 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:37.609120 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 807/1624] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:21:37 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:21:37.611953 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21ca5f-3375-4b72-bab3-32de517d5f6a service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:37.629180 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21ca5f-3375-4b72-bab3-32de517d5f6a service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:37.645904 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa9480-789c-49bf-bc2a-098f290a7b7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:37.646604 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa9480-789c-49bf-bc2a-098f290a7b7a service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:37.646896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa9480-789c-49bf-bc2a-098f290a7b7a service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:37.647207 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa9480-789c-49bf-bc2a-098f290a7b7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:37.671911 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:37.748771 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e95af5-746e-4d2f-a16b-73c012331149 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:37.752126 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e95af5-746e-4d2f-a16b-73c012331149 service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:37.861724 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 818/1625] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:21:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:21:37.861724 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e95af5-746e-4d2f-a16b-73c012331149 service nova] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:37.862692 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fa9480-789c-49bf-bc2a-098f290a7b7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:37.862692 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fa9480-789c-49bf-bc2a-098f290a7b7a service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:37.863961 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e95af5-746e-4d2f-a16b-73c012331149 service nova] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:37.879477 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898c0c1-8da5-4b14-a0e7-fd73e2e0a411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:37.881224 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898c0c1-8da5-4b14-a0e7-fd73e2e0a411 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:37.881608 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898c0c1-8da5-4b14-a0e7-fd73e2e0a411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:37.882586 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898c0c1-8da5-4b14-a0e7-fd73e2e0a411 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:37.883972 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898c0c1-8da5-4b14-a0e7-fd73e2e0a411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:37.886569 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 808/1626] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:21:37 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:21:37.890647 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa9480-789c-49bf-bc2a-098f290a7b7a service nova] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:37.913825 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898c0c1-8da5-4b14-a0e7-fd73e2e0a411 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:37.938828 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa9480-789c-49bf-bc2a-098f290a7b7a service nova] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:37.958095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393adb7e-bf45-4041-8df5-6b898f7d1182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:37.959082 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393adb7e-bf45-4041-8df5-6b898f7d1182 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:37.959520 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393adb7e-bf45-4041-8df5-6b898f7d1182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:37.959965 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393adb7e-bf45-4041-8df5-6b898f7d1182 admin admin] PATH_INFO: `/v3/projects/fac2034a19a94baea512aad71fe1a111/users/a7a30ae6358f48ac8559388ab864e580/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:37.966303 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 819/1627] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:37 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:37.968546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898c0c1-8da5-4b14-a0e7-fd73e2e0a411 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:38.012204 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898c0c1-8da5-4b14-a0e7-fd73e2e0a411 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:38.034088 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393adb7e-bf45-4041-8df5-6b898f7d1182 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fac2034a19a94baea512aad71fe1a111, user_id=a7a30ae6358f48ac8559388ab864e580, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:38.034911 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f17940-7d1e-4960-bdc7-00e38179f388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:38.036362 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393adb7e-bf45-4041-8df5-6b898f7d1182 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:38.036585 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f17940-7d1e-4960-bdc7-00e38179f388 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:38.037265 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f17940-7d1e-4960-bdc7-00e38179f388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:38.039695 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f17940-7d1e-4960-bdc7-00e38179f388 admin admin] PATH_INFO: `/v3/projects/9575777fe13344479a4a4948ca3ce945/users/2ce1af04ef934727845e63742ea867ad/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:38.082894 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 809/1628] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:21:37 2022] PUT /identity/v3/projects/fac2034a19a94baea512aad71fe1a111/users/a7a30ae6358f48ac8559388ab864e580/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:38.088978 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393adb7e-bf45-4041-8df5-6b898f7d1182 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:38.101464 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f17940-7d1e-4960-bdc7-00e38179f388 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9575777fe13344479a4a4948ca3ce945, user_id=2ce1af04ef934727845e63742ea867ad, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:38.113582 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f17940-7d1e-4960-bdc7-00e38179f388 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:38.157648 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 820/1629] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:21:37 2022] PUT /identity/v3/projects/9575777fe13344479a4a4948ca3ce945/users/2ce1af04ef934727845e63742ea867ad/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:38.163461 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f17940-7d1e-4960-bdc7-00e38179f388 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:38.182844 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:38.258781 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393adb7e-bf45-4041-8df5-6b898f7d1182 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:38.396636 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f17940-7d1e-4960-bdc7-00e38179f388 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:38.412728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b286f5e-004b-42ac-90fd-782663a29171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:38.413569 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b286f5e-004b-42ac-90fd-782663a29171 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:38.413965 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b286f5e-004b-42ac-90fd-782663a29171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:38.414373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b286f5e-004b-42ac-90fd-782663a29171 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:38.414679 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba2a96-d65a-43b8-a663-282c08b1680c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:38.415307 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b286f5e-004b-42ac-90fd-782663a29171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:38.415433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba2a96-d65a-43b8-a663-282c08b1680c admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:38.415592 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba2a96-d65a-43b8-a663-282c08b1680c admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:38.415864 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba2a96-d65a-43b8-a663-282c08b1680c admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:38.416673 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cba2a96-d65a-43b8-a663-282c08b1680c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:38.421324 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cba2a96-d65a-43b8-a663-282c08b1680c admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:38.424302 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b286f5e-004b-42ac-90fd-782663a29171 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:38.458149 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 810/1630] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:38.459416 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba2a96-d65a-43b8-a663-282c08b1680c admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:38.472288 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 821/1631] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:38.473624 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b286f5e-004b-42ac-90fd-782663a29171 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:38.487549 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba2a96-d65a-43b8-a663-282c08b1680c admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:38.502637 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b286f5e-004b-42ac-90fd-782663a29171 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:38.503230 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a2475-c2aa-4780-bdc9-e56d6bc71589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:38.505383 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a2475-c2aa-4780-bdc9-e56d6bc71589 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:38.506698 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a2475-c2aa-4780-bdc9-e56d6bc71589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:38.509251 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a2475-c2aa-4780-bdc9-e56d6bc71589 admin admin] PATH_INFO: `/v3/projects/fac2034a19a94baea512aad71fe1a111/users/a7a30ae6358f48ac8559388ab864e580/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:38.512938 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128a2475-c2aa-4780-bdc9-e56d6bc71589 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fac2034a19a94baea512aad71fe1a111, user_id=a7a30ae6358f48ac8559388ab864e580, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:38.520364 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128a2475-c2aa-4780-bdc9-e56d6bc71589 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:38.520920 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28270237-d8e5-4e2a-b35d-c5382ea510e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:38.521796 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28270237-d8e5-4e2a-b35d-c5382ea510e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:38.522455 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28270237-d8e5-4e2a-b35d-c5382ea510e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:38.522889 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28270237-d8e5-4e2a-b35d-c5382ea510e2 admin admin] PATH_INFO: `/v3/projects/9575777fe13344479a4a4948ca3ce945/users/2ce1af04ef934727845e63742ea867ad/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:38.529228 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28270237-d8e5-4e2a-b35d-c5382ea510e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9575777fe13344479a4a4948ca3ce945, user_id=2ce1af04ef934727845e63742ea867ad, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:38.534542 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28270237-d8e5-4e2a-b35d-c5382ea510e2 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:38.553501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 822/1632] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:21:38 2022] PUT /identity/v3/projects/9575777fe13344479a4a4948ca3ce945/users/2ce1af04ef934727845e63742ea867ad/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:21:38.564416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2ca0c-7355-44a4-b800-001c0c0cad38 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:38.564970 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2ca0c-7355-44a4-b800-001c0c0cad38 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:38.565403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2ca0c-7355-44a4-b800-001c0c0cad38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:38.574193 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 811/1633] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:21:38 2022] PUT /identity/v3/projects/fac2034a19a94baea512aad71fe1a111/users/a7a30ae6358f48ac8559388ab864e580/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:38.575738 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd7481-9687-41f3-a183-f4270aa5ff0a None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:38.576023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd7481-9687-41f3-a183-f4270aa5ff0a None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:38.576300 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd7481-9687-41f3-a183-f4270aa5ff0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:38.650838 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-eaf2ca0c-7355-44a4-b800-001c0c0cad38 None None] MFA Rules not processed for user `2ce1af04ef934727845e63742ea867ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:38.654858 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-eaf2ca0c-7355-44a4-b800-001c0c0cad38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:38.694947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-f0bd7481-9687-41f3-a183-f4270aa5ff0a None None] MFA Rules not processed for user `a7a30ae6358f48ac8559388ab864e580`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:38.698991 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f0bd7481-9687-41f3-a183-f4270aa5ff0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:38.821475 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 823/1634] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:38 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:38.833353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a758eed-f6e8-442d-acc5-493e2808f914 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:38.840172 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a758eed-f6e8-442d-acc5-493e2808f914 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:38.840809 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a758eed-f6e8-442d-acc5-493e2808f914 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:38.844258 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 824/1635] 10.222.0.46 () {54 vars in 912 bytes} [Wed Jul 27 04:21:38 2022] GET /identity => generated 270 bytes in 22 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:21:38.845308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a758eed-f6e8-442d-acc5-493e2808f914 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:38.856463 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:38.952616 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 812/1636] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:38 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:38.954037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426d2af-46b7-4385-8697-1dad23a6274d None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:38.954332 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426d2af-46b7-4385-8697-1dad23a6274d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:38.954605 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426d2af-46b7-4385-8697-1dad23a6274d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:39.005945 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a758eed-f6e8-442d-acc5-493e2808f914 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:39.011925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4535dbf-015c-44ab-9b07-6f182d4e2de9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:39.012481 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4535dbf-015c-44ab-9b07-6f182d4e2de9 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:39.012716 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4535dbf-015c-44ab-9b07-6f182d4e2de9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:39.012988 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4535dbf-015c-44ab-9b07-6f182d4e2de9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:39.021953 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-b426d2af-46b7-4385-8697-1dad23a6274d None None] MFA Rules not processed for user `9dc9ea8f89c14a2a8e3272f750647dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:39.024112 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4535dbf-015c-44ab-9b07-6f182d4e2de9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:39.024313 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b426d2af-46b7-4385-8697-1dad23a6274d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:39.025496 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4535dbf-015c-44ab-9b07-6f182d4e2de9 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:39.039797 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 825/1637] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:21:38 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:21:39.051523 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182c6c4-5291-4ca4-a6fe-4f69f6695299 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:39.051814 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182c6c4-5291-4ca4-a6fe-4f69f6695299 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:39.052097 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182c6c4-5291-4ca4-a6fe-4f69f6695299 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:39.064165 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 826/1638] 10.222.0.46 () {54 vars in 912 bytes} [Wed Jul 27 04:21:39 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:21:39.067106 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182c6c4-5291-4ca4-a6fe-4f69f6695299 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:39.083058 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182c6c4-5291-4ca4-a6fe-4f69f6695299 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:39.093029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438c86d-f75c-4779-941a-6d06465e8bb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:39.095636 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438c86d-f75c-4779-941a-6d06465e8bb2 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:39.097648 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438c86d-f75c-4779-941a-6d06465e8bb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:39.097942 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438c86d-f75c-4779-941a-6d06465e8bb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:39.122048 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438c86d-f75c-4779-941a-6d06465e8bb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:39.133013 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438c86d-f75c-4779-941a-6d06465e8bb2 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:39.134197 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:39.163710 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 827/1639] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:21:39 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:21:39.167381 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e9acf-3ef9-4246-997e-60ef660463c6 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:39.167862 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e9acf-3ef9-4246-997e-60ef660463c6 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:39.169398 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e9acf-3ef9-4246-997e-60ef660463c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:39.189840 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 813/1640] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Jul 27 04:21:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:39.194730 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e52aa-7dc9-4b79-8814-678ed86a6664 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:39.194730 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e52aa-7dc9-4b79-8814-678ed86a6664 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:39.195014 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e52aa-7dc9-4b79-8814-678ed86a6664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:39.237390 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-da7e9acf-3ef9-4246-997e-60ef660463c6 None None] MFA Rules not processed for user `97602bd0c5ce4e11819b634995d93405`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:39.238204 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-da7e9acf-3ef9-4246-997e-60ef660463c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:39.242129 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-c46e52aa-7dc9-4b79-8814-678ed86a6664 None None] MFA Rules not processed for user `9dc9ea8f89c14a2a8e3272f750647dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:39.244384 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c46e52aa-7dc9-4b79-8814-678ed86a6664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:39.270810 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 814/1641] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Jul 27 04:21:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:39.271682 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46e52aa-7dc9-4b79-8814-678ed86a6664 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:39.296271 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46e52aa-7dc9-4b79-8814-678ed86a6664 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:39.308911 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d542743-37ed-4031-b7b4-d742660e89ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:39.309565 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d542743-37ed-4031-b7b4-d742660e89ed service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:39.309847 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d542743-37ed-4031-b7b4-d742660e89ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:39.310150 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d542743-37ed-4031-b7b4-d742660e89ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:39.332945 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d542743-37ed-4031-b7b4-d742660e89ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:39.334423 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d542743-37ed-4031-b7b4-d742660e89ed service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:39.362504 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 815/1642] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:21:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:21:39.408436 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 828/1643] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:39 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:39.413230 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d542743-37ed-4031-b7b4-d742660e89ed service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:39.485425 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e9acf-3ef9-4246-997e-60ef660463c6 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:39.503854 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e9acf-3ef9-4246-997e-60ef660463c6 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:39.512676 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c865602a-1afb-467c-b8f1-6fcaa60da156 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:39.514128 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865602a-1afb-467c-b8f1-6fcaa60da156 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:39.514587 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865602a-1afb-467c-b8f1-6fcaa60da156 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:39.515024 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865602a-1afb-467c-b8f1-6fcaa60da156 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:39.534602 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c865602a-1afb-467c-b8f1-6fcaa60da156 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:39.537478 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c865602a-1afb-467c-b8f1-6fcaa60da156 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:39.551014 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d542743-37ed-4031-b7b4-d742660e89ed service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:39.566397 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2cdc30-de4e-4d42-b41f-ff981a5c5e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:39.567186 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2cdc30-de4e-4d42-b41f-ff981a5c5e90 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:39.567552 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2cdc30-de4e-4d42-b41f-ff981a5c5e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:39.567887 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2cdc30-de4e-4d42-b41f-ff981a5c5e90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:39.568699 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2cdc30-de4e-4d42-b41f-ff981a5c5e90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:39.571277 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2cdc30-de4e-4d42-b41f-ff981a5c5e90 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:39.573700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 829/1644] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:21:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 04:21:39.644612 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 816/1645] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:21:39 2022] POST /identity/v3/projects => generated 405 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:39.649174 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c865602a-1afb-467c-b8f1-6fcaa60da156 service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:39.669940 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c865602a-1afb-467c-b8f1-6fcaa60da156 service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:39.801708 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2cdc30-de4e-4d42-b41f-ff981a5c5e90 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:39.935767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c02952-bdc7-4ebb-8ee6-9838d985a7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:39.936426 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c02952-bdc7-4ebb-8ee6-9838d985a7a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:39.936716 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c02952-bdc7-4ebb-8ee6-9838d985a7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:39.936995 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c02952-bdc7-4ebb-8ee6-9838d985a7a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:39.937768 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c02952-bdc7-4ebb-8ee6-9838d985a7a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:39.950527 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c02952-bdc7-4ebb-8ee6-9838d985a7a1 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:39.991134 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:39.997819 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2cdc30-de4e-4d42-b41f-ff981a5c5e90 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:40.006142 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 830/1646] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:21:39 2022] POST /identity/v3/users => generated 309 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:40.007324 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c02952-bdc7-4ebb-8ee6-9838d985a7a1 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:40.016909 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14d1f0-3839-47c2-8563-476c3ab7a7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:40.017702 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14d1f0-3839-47c2-8563-476c3ab7a7d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:40.020774 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14d1f0-3839-47c2-8563-476c3ab7a7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:40.021125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14d1f0-3839-47c2-8563-476c3ab7a7d8 admin admin] PATH_INFO: `/v3/users/90c723ce0dec415494fe64c7e03b2915` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:40.022662 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e14d1f0-3839-47c2-8563-476c3ab7a7d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90c723ce0dec415494fe64c7e03b2915)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:40.040756 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e14d1f0-3839-47c2-8563-476c3ab7a7d8 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:40.189062 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c02952-bdc7-4ebb-8ee6-9838d985a7a1 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:40.196281 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1e14d1f0-3839-47c2-8563-476c3ab7a7d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90c723ce0dec415494fe64c7e03b2915'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:40.197227 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1e14d1f0-3839-47c2-8563-476c3ab7a7d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90c723ce0dec415494fe64c7e03b2915'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:40.198657 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c426f0e-aed6-4cd4-b6da-170a9bc71c14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:40.199322 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c426f0e-aed6-4cd4-b6da-170a9bc71c14 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:40.199607 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c426f0e-aed6-4cd4-b6da-170a9bc71c14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:40.199887 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c426f0e-aed6-4cd4-b6da-170a9bc71c14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:40.203061 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:40.233963 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1e14d1f0-3839-47c2-8563-476c3ab7a7d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90c723ce0dec415494fe64c7e03b2915'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:40.264730 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1e14d1f0-3839-47c2-8563-476c3ab7a7d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90c723ce0dec415494fe64c7e03b2915'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:40.287782 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1e14d1f0-3839-47c2-8563-476c3ab7a7d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90c723ce0dec415494fe64c7e03b2915'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:40.315299 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 817/1647] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:21:39 2022] DELETE /identity/v3/users/90c723ce0dec415494fe64c7e03b2915 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:21:40.318499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14d1f0-3839-47c2-8563-476c3ab7a7d8 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:40.324747 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-1e14d1f0-3839-47c2-8563-476c3ab7a7d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:40.393881 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c426f0e-aed6-4cd4-b6da-170a9bc71c14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:40.395407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c426f0e-aed6-4cd4-b6da-170a9bc71c14 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:40.415499 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 831/1648] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:21:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:21:40.418211 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c426f0e-aed6-4cd4-b6da-170a9bc71c14 service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:40.458710 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c426f0e-aed6-4cd4-b6da-170a9bc71c14 service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:40.567965 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14d1f0-3839-47c2-8563-476c3ab7a7d8 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:40.582839 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c7d08-c822-48a5-9bd9-45c987128eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:40.583643 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c7d08-c822-48a5-9bd9-45c987128eae admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:40.584466 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c7d08-c822-48a5-9bd9-45c987128eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:40.586126 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c7d08-c822-48a5-9bd9-45c987128eae admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:40.587016 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4c7d08-c822-48a5-9bd9-45c987128eae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:40.587788 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aed099-1348-49cf-939e-5302231ed3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:40.589376 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aed099-1348-49cf-939e-5302231ed3d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:40.589862 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aed099-1348-49cf-939e-5302231ed3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:40.590321 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aed099-1348-49cf-939e-5302231ed3d3 admin admin] PATH_INFO: `/v3/users/d2b422350e7d4995ba31ccefa9333dcf` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:40.591256 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4c7d08-c822-48a5-9bd9-45c987128eae admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:40.592165 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aed099-1348-49cf-939e-5302231ed3d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2b422350e7d4995ba31ccefa9333dcf)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:40.599014 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: WARNING py.warnings [None req-a6aed099-1348-49cf-939e-5302231ed3d3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 04:21:40.599014 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: warnings.warn(msg) Jul 27 04:21:40.599014 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: Jul 27 04:21:40.605413 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aed099-1348-49cf-939e-5302231ed3d3 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:40.617899 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 818/1649] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:40.622015 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ced96-fb53-488d-8a2b-3cfa8a032031 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:40.622384 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ced96-fb53-488d-8a2b-3cfa8a032031 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:40.623761 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ced96-fb53-488d-8a2b-3cfa8a032031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:40.668276 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-ae1ced96-fb53-488d-8a2b-3cfa8a032031 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:40.669778 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ae1ced96-fb53-488d-8a2b-3cfa8a032031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:40.693530 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 819/1650] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:40.698297 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1ced96-fb53-488d-8a2b-3cfa8a032031 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:40.728725 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1ced96-fb53-488d-8a2b-3cfa8a032031 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:40.744447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a6aed099-1348-49cf-939e-5302231ed3d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2b422350e7d4995ba31ccefa9333dcf'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:40.747051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a6aed099-1348-49cf-939e-5302231ed3d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2b422350e7d4995ba31ccefa9333dcf'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:40.747397 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18487ce6-3d69-4758-94e6-57e3352ba3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:40.748023 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a6aed099-1348-49cf-939e-5302231ed3d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2b422350e7d4995ba31ccefa9333dcf'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:40.748382 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18487ce6-3d69-4758-94e6-57e3352ba3c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:40.748771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18487ce6-3d69-4758-94e6-57e3352ba3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:40.750343 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18487ce6-3d69-4758-94e6-57e3352ba3c3 admin admin] PATH_INFO: `/v3/projects/d9f6e6209ad24963b27eb9a6aabbe092/users/c2cd62a35ffb4338a6387ce793622407/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:40.771435 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:40.798394 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a6aed099-1348-49cf-939e-5302231ed3d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2b422350e7d4995ba31ccefa9333dcf'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:40.799049 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18487ce6-3d69-4758-94e6-57e3352ba3c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9f6e6209ad24963b27eb9a6aabbe092, user_id=c2cd62a35ffb4338a6387ce793622407, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:40.804868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18487ce6-3d69-4758-94e6-57e3352ba3c3 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:40.842065 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 820/1651] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:21:40 2022] PUT /identity/v3/projects/d9f6e6209ad24963b27eb9a6aabbe092/users/c2cd62a35ffb4338a6387ce793622407/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:40.843036 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18487ce6-3d69-4758-94e6-57e3352ba3c3 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:40.847812 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a6aed099-1348-49cf-939e-5302231ed3d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2b422350e7d4995ba31ccefa9333dcf'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:40.853910 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-18487ce6-3d69-4758-94e6-57e3352ba3c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:40.881360 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 832/1652] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:21:40 2022] DELETE /identity/v3/users/d2b422350e7d4995ba31ccefa9333dcf => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:40.882415 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aed099-1348-49cf-939e-5302231ed3d3 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:40.884258 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a6aed099-1348-49cf-939e-5302231ed3d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:41.023999 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aed099-1348-49cf-939e-5302231ed3d3 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:41.033140 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c08575-411c-4cb5-9148-9e762d0003b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:41.034096 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c08575-411c-4cb5-9148-9e762d0003b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:41.034547 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c08575-411c-4cb5-9148-9e762d0003b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:41.038122 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18487ce6-3d69-4758-94e6-57e3352ba3c3 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:41.054584 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c08575-411c-4cb5-9148-9e762d0003b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:41.055453 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c08575-411c-4cb5-9148-9e762d0003b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:41.061358 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c08575-411c-4cb5-9148-9e762d0003b9 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:41.093387 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 833/1653] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:41.095775 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ea4c6-7746-476c-8264-5a7a68a1b434 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:41.096079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ea4c6-7746-476c-8264-5a7a68a1b434 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:41.096362 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ea4c6-7746-476c-8264-5a7a68a1b434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:41.155744 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-826ea4c6-7746-476c-8264-5a7a68a1b434 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:41.157205 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-826ea4c6-7746-476c-8264-5a7a68a1b434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:41.187355 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:41.190237 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 834/1654] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:41.191394 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ea4c6-7746-476c-8264-5a7a68a1b434 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:41.212297 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54021e86-b040-41fc-ab48-554114694967 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:41.213023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54021e86-b040-41fc-ab48-554114694967 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:41.213244 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54021e86-b040-41fc-ab48-554114694967 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:41.213525 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54021e86-b040-41fc-ab48-554114694967 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:41.220553 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ea4c6-7746-476c-8264-5a7a68a1b434 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:41.221799 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-54021e86-b040-41fc-ab48-554114694967 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:41.234601 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67febbaa-27e9-4fc8-be45-1f322b210032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:41.236500 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67febbaa-27e9-4fc8-be45-1f322b210032 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:41.237357 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67febbaa-27e9-4fc8-be45-1f322b210032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:41.239582 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67febbaa-27e9-4fc8-be45-1f322b210032 admin admin] PATH_INFO: `/v3/projects/d9f6e6209ad24963b27eb9a6aabbe092/users/c2cd62a35ffb4338a6387ce793622407/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:41.243731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67febbaa-27e9-4fc8-be45-1f322b210032 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9f6e6209ad24963b27eb9a6aabbe092, user_id=c2cd62a35ffb4338a6387ce793622407, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:41.244808 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54021e86-b040-41fc-ab48-554114694967 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:41.247124 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54021e86-b040-41fc-ab48-554114694967 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:41.261084 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67febbaa-27e9-4fc8-be45-1f322b210032 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:41.264917 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 821/1655] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:21:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:21:41.266171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54021e86-b040-41fc-ab48-554114694967 service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:41.268352 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-54021e86-b040-41fc-ab48-554114694967 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:41.282135 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54021e86-b040-41fc-ab48-554114694967 service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:41.300561 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 835/1656] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:21:41 2022] PUT /identity/v3/projects/d9f6e6209ad24963b27eb9a6aabbe092/users/c2cd62a35ffb4338a6387ce793622407/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:41.347611 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c387e38-30ff-42dd-8922-612e231c21f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:41.348716 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c387e38-30ff-42dd-8922-612e231c21f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:41.349212 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c387e38-30ff-42dd-8922-612e231c21f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:41.349627 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c387e38-30ff-42dd-8922-612e231c21f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:41.472945 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c387e38-30ff-42dd-8922-612e231c21f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:41.476539 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c387e38-30ff-42dd-8922-612e231c21f1 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:41.499252 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 822/1657] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:21:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:21:42.403013 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67febbaa-27e9-4fc8-be45-1f322b210032 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:42.404833 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-67febbaa-27e9-4fc8-be45-1f322b210032 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:42.428592 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67febbaa-27e9-4fc8-be45-1f322b210032 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:42.459090 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7da4e26-1b19-4336-a504-6826e1f4e65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:42.459408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da4e26-1b19-4336-a504-6826e1f4e65e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:42.459700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da4e26-1b19-4336-a504-6826e1f4e65e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:42.459978 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da4e26-1b19-4336-a504-6826e1f4e65e admin admin] PATH_INFO: `/v3/projects/92b1920cfcbe49ea9f9a65d9e1c2f5fd` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:42.462914 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7da4e26-1b19-4336-a504-6826e1f4e65e admin admin] RBAC: Authorizing `identity:delete_project(project_id=92b1920cfcbe49ea9f9a65d9e1c2f5fd)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:42.473144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7da4e26-1b19-4336-a504-6826e1f4e65e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:42.645403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d7da4e26-1b19-4336-a504-6826e1f4e65e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92b1920cfcbe49ea9f9a65d9e1c2f5fd', 'request_id': 'req-d7da4e26-1b19-4336-a504-6826e1f4e65e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:42.667122 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d7da4e26-1b19-4336-a504-6826e1f4e65e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92b1920cfcbe49ea9f9a65d9e1c2f5fd', 'request_id': 'req-d7da4e26-1b19-4336-a504-6826e1f4e65e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:42.693499 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d7da4e26-1b19-4336-a504-6826e1f4e65e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92b1920cfcbe49ea9f9a65d9e1c2f5fd', 'request_id': 'req-d7da4e26-1b19-4336-a504-6826e1f4e65e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:42.728545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d7da4e26-1b19-4336-a504-6826e1f4e65e admin admin] The token cache is being invalidate because project 92b1920cfcbe49ea9f9a65d9e1c2f5fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:21:42.729394 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d7da4e26-1b19-4336-a504-6826e1f4e65e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:42.730338 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d7da4e26-1b19-4336-a504-6826e1f4e65e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:42.742332 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 836/1658] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:21:42 2022] DELETE /identity/v3/projects/92b1920cfcbe49ea9f9a65d9e1c2f5fd => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:42.746376 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c387e38-30ff-42dd-8922-612e231c21f1 service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:42.758015 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c387e38-30ff-42dd-8922-612e231c21f1 service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:42.946049 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe80c11-6cad-4389-bb13-ba739687309b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:42.950205 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe80c11-6cad-4389-bb13-ba739687309b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:42.950903 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe80c11-6cad-4389-bb13-ba739687309b admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:42.951432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe80c11-6cad-4389-bb13-ba739687309b admin admin] PATH_INFO: `/v3/users/2ce1af04ef934727845e63742ea867ad` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:42.956127 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe80c11-6cad-4389-bb13-ba739687309b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ce1af04ef934727845e63742ea867ad)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:42.958817 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe80c11-6cad-4389-bb13-ba739687309b admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:43.058821 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:43.082191 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-cbe80c11-6cad-4389-bb13-ba739687309b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ce1af04ef934727845e63742ea867ad'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:43.084036 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-cbe80c11-6cad-4389-bb13-ba739687309b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ce1af04ef934727845e63742ea867ad'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:43.109348 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-cbe80c11-6cad-4389-bb13-ba739687309b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ce1af04ef934727845e63742ea867ad'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:43.146242 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-cbe80c11-6cad-4389-bb13-ba739687309b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ce1af04ef934727845e63742ea867ad'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:43.176295 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-cbe80c11-6cad-4389-bb13-ba739687309b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ce1af04ef934727845e63742ea867ad'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:43.186020 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 823/1659] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:21:42 2022] DELETE /identity/v3/users/2ce1af04ef934727845e63742ea867ad => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:43.669756 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7da4e26-1b19-4336-a504-6826e1f4e65e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:43.671709 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d7da4e26-1b19-4336-a504-6826e1f4e65e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:43.711252 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7da4e26-1b19-4336-a504-6826e1f4e65e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:43.772977 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:43.825852 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d85cdf6-0deb-4cda-863e-ed3f55b43167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:43.827521 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85cdf6-0deb-4cda-863e-ed3f55b43167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:43.827962 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85cdf6-0deb-4cda-863e-ed3f55b43167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:43.828433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85cdf6-0deb-4cda-863e-ed3f55b43167 admin admin] PATH_INFO: `/v3/projects/b97850dff881499c9c9d6a66d43adfb4` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:43.829824 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d85cdf6-0deb-4cda-863e-ed3f55b43167 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b97850dff881499c9c9d6a66d43adfb4)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:43.831905 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d85cdf6-0deb-4cda-863e-ed3f55b43167 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:43.961699 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0d85cdf6-0deb-4cda-863e-ed3f55b43167 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b97850dff881499c9c9d6a66d43adfb4', 'request_id': 'req-0d85cdf6-0deb-4cda-863e-ed3f55b43167'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:43.989076 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0d85cdf6-0deb-4cda-863e-ed3f55b43167 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b97850dff881499c9c9d6a66d43adfb4', 'request_id': 'req-0d85cdf6-0deb-4cda-863e-ed3f55b43167'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:44.022285 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0d85cdf6-0deb-4cda-863e-ed3f55b43167 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b97850dff881499c9c9d6a66d43adfb4', 'request_id': 'req-0d85cdf6-0deb-4cda-863e-ed3f55b43167'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:44.070324 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0d85cdf6-0deb-4cda-863e-ed3f55b43167 admin admin] The token cache is being invalidate because project b97850dff881499c9c9d6a66d43adfb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:21:44.070786 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0d85cdf6-0deb-4cda-863e-ed3f55b43167 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:44.075150 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0d85cdf6-0deb-4cda-863e-ed3f55b43167 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:44.083683 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 837/1660] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:21:43 2022] DELETE /identity/v3/projects/b97850dff881499c9c9d6a66d43adfb4 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:44.088640 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49847904-fb1e-4e8d-acd7-01481a22b454 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:44.089170 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49847904-fb1e-4e8d-acd7-01481a22b454 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:44.089611 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49847904-fb1e-4e8d-acd7-01481a22b454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:44.129953 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-49847904-fb1e-4e8d-acd7-01481a22b454 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:44.131287 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-49847904-fb1e-4e8d-acd7-01481a22b454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:44.354681 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 824/1661] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:44.357049 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd21780-e904-45bb-bc26-1c542abc012a None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:44.357382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd21780-e904-45bb-bc26-1c542abc012a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:44.357637 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd21780-e904-45bb-bc26-1c542abc012a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:44.400488 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-5dd21780-e904-45bb-bc26-1c542abc012a None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:44.402840 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5dd21780-e904-45bb-bc26-1c542abc012a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:44.417284 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 838/1662] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:44.421453 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49847904-fb1e-4e8d-acd7-01481a22b454 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:44.435482 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49847904-fb1e-4e8d-acd7-01481a22b454 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:44.452421 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e82ae-8727-4d40-8660-874c5e83711f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:44.453327 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e82ae-8727-4d40-8660-874c5e83711f admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:44.453789 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e82ae-8727-4d40-8660-874c5e83711f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:44.454264 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e82ae-8727-4d40-8660-874c5e83711f admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:44.455171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8e82ae-8727-4d40-8660-874c5e83711f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:21:44.455931 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8e82ae-8727-4d40-8660-874c5e83711f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:44.457769 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8e82ae-8727-4d40-8660-874c5e83711f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:44.477540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 825/1663] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:21:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:21:44.481793 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd21780-e904-45bb-bc26-1c542abc012a None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:44.492525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd21780-e904-45bb-bc26-1c542abc012a None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:44.500342 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e8f82-dbc4-418a-b717-e40bd4bfd01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:44.501109 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e8f82-dbc4-418a-b717-e40bd4bfd01c admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:44.501508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e8f82-dbc4-418a-b717-e40bd4bfd01c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:44.501899 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e8f82-dbc4-418a-b717-e40bd4bfd01c admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:44.502737 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9e8f82-dbc4-418a-b717-e40bd4bfd01c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:44.507421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9e8f82-dbc4-418a-b717-e40bd4bfd01c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:44.578018 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 839/1664] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:21:44 2022] POST /identity/v3/projects => generated 377 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:44.579465 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e82ae-8727-4d40-8660-874c5e83711f admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:44.596633 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e82ae-8727-4d40-8660-874c5e83711f admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:44.717393 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e733eb-3104-4c8f-b9c5-2c25effecf09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:44.718059 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e733eb-3104-4c8f-b9c5-2c25effecf09 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:44.718343 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e733eb-3104-4c8f-b9c5-2c25effecf09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:44.718619 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e733eb-3104-4c8f-b9c5-2c25effecf09 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:44.719321 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e733eb-3104-4c8f-b9c5-2c25effecf09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:44.722203 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e733eb-3104-4c8f-b9c5-2c25effecf09 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:44.785275 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 826/1665] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:21:44 2022] POST /identity/v3/users => generated 289 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:21:44.785884 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e8f82-dbc4-418a-b717-e40bd4bfd01c admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:44.800701 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e8f82-dbc4-418a-b717-e40bd4bfd01c admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:44.813028 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee1903-43da-4ba6-8df8-4a300d6aff79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:44.813692 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee1903-43da-4ba6-8df8-4a300d6aff79 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:44.814003 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee1903-43da-4ba6-8df8-4a300d6aff79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:44.814292 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee1903-43da-4ba6-8df8-4a300d6aff79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:44.815085 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ee1903-43da-4ba6-8df8-4a300d6aff79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:44.816723 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ee1903-43da-4ba6-8df8-4a300d6aff79 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:44.847368 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 840/1666] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:44.851369 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e733eb-3104-4c8f-b9c5-2c25effecf09 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:44.871178 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e733eb-3104-4c8f-b9c5-2c25effecf09 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:44.887599 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da749b-d324-4470-9f32-c5f88b96bd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:44.888245 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da749b-d324-4470-9f32-c5f88b96bd4d admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:44.888769 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da749b-d324-4470-9f32-c5f88b96bd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:44.888769 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da749b-d324-4470-9f32-c5f88b96bd4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:44.889487 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da749b-d324-4470-9f32-c5f88b96bd4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:44.891127 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da749b-d324-4470-9f32-c5f88b96bd4d admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:44.927486 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 827/1667] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:44.928097 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee1903-43da-4ba6-8df8-4a300d6aff79 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:44.950414 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee1903-43da-4ba6-8df8-4a300d6aff79 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:44.962723 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20434c1-c44d-4538-aa11-afb237fd43bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:44.964585 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20434c1-c44d-4538-aa11-afb237fd43bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:44.964770 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20434c1-c44d-4538-aa11-afb237fd43bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:44.965398 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20434c1-c44d-4538-aa11-afb237fd43bc admin admin] PATH_INFO: `/v3/projects/ba694613897b4442bf3a82f72a75a8b3/users/e71d2e81acb048beb116c8b31fe68a56/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:45.000994 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20434c1-c44d-4538-aa11-afb237fd43bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba694613897b4442bf3a82f72a75a8b3, user_id=e71d2e81acb048beb116c8b31fe68a56, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:45.004081 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20434c1-c44d-4538-aa11-afb237fd43bc admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:45.036583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 841/1668] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:21:44 2022] PUT /identity/v3/projects/ba694613897b4442bf3a82f72a75a8b3/users/e71d2e81acb048beb116c8b31fe68a56/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:21:45.052513 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6d27c-31f6-4455-a1c8-79b0fbd5b384 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:45.052513 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6d27c-31f6-4455-a1c8-79b0fbd5b384 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:45.052513 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6d27c-31f6-4455-a1c8-79b0fbd5b384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:45.090902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-4cb6d27c-31f6-4455-a1c8-79b0fbd5b384 None None] MFA Rules not processed for user `e71d2e81acb048beb116c8b31fe68a56`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:45.091494 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-4cb6d27c-31f6-4455-a1c8-79b0fbd5b384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:45.111006 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:45.205755 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 828/1669] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:45.206662 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20434c1-c44d-4538-aa11-afb237fd43bc admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:45.243102 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4995a-bae1-4a57-b10a-3f4d19cb520f None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:45.244006 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4995a-bae1-4a57-b10a-3f4d19cb520f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:45.245696 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4995a-bae1-4a57-b10a-3f4d19cb520f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:45.337500 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-a0b4995a-bae1-4a57-b10a-3f4d19cb520f None None] MFA Rules not processed for user `c2cd62a35ffb4338a6387ce793622407`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:45.338904 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-a0b4995a-bae1-4a57-b10a-3f4d19cb520f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:45.350825 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:45.359434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20434c1-c44d-4538-aa11-afb237fd43bc admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:45.375966 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938cde4f-767e-4c78-a5c2-b4e482b74dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:45.376731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938cde4f-767e-4c78-a5c2-b4e482b74dd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:45.377058 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938cde4f-767e-4c78-a5c2-b4e482b74dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:45.378397 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938cde4f-767e-4c78-a5c2-b4e482b74dd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:45.379160 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938cde4f-767e-4c78-a5c2-b4e482b74dd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:45.382223 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938cde4f-767e-4c78-a5c2-b4e482b74dd5 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:45.470777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 842/1670] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:21:45 2022] POST /identity/v3/projects => generated 377 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:45.483477 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938cde4f-767e-4c78-a5c2-b4e482b74dd5 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:45.570020 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 829/1671] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:45 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:45.571897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7da07-f789-4ecd-9daf-eb0e79bcf9f2 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:45.576348 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7da07-f789-4ecd-9daf-eb0e79bcf9f2 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:45.578262 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7da07-f789-4ecd-9daf-eb0e79bcf9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:45.654053 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-4cc7da07-f789-4ecd-9daf-eb0e79bcf9f2 None None] MFA Rules not processed for user `5a69202d87094049b12765d64c17e515`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:45.655783 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-4cc7da07-f789-4ecd-9daf-eb0e79bcf9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:45.693900 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938cde4f-767e-4c78-a5c2-b4e482b74dd5 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:45.702994 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b9724a-86ba-419b-83ad-7ebc9e2e2036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:45.703701 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9724a-86ba-419b-83ad-7ebc9e2e2036 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:45.703988 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9724a-86ba-419b-83ad-7ebc9e2e2036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:45.704260 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9724a-86ba-419b-83ad-7ebc9e2e2036 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:45.705122 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b9724a-86ba-419b-83ad-7ebc9e2e2036 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:45.708176 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b9724a-86ba-419b-83ad-7ebc9e2e2036 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:45.765026 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 843/1672] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:21:45 2022] POST /identity/v3/users => generated 295 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:45.770436 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b9724a-86ba-419b-83ad-7ebc9e2e2036 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:45.773383 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-05b9724a-86ba-419b-83ad-7ebc9e2e2036 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:45.822255 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 830/1673] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:45.822400 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b9724a-86ba-419b-83ad-7ebc9e2e2036 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:45.825063 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc7da07-f789-4ecd-9daf-eb0e79bcf9f2 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:45.840773 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3e147-f359-4cf6-8587-fd7d0d14c71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:45.842604 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3e147-f359-4cf6-8587-fd7d0d14c71c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:45.842958 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3e147-f359-4cf6-8587-fd7d0d14c71c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:45.843153 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3e147-f359-4cf6-8587-fd7d0d14c71c admin admin] PATH_INFO: `/v3/projects/9575777fe13344479a4a4948ca3ce945` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:45.846396 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a3e147-f359-4cf6-8587-fd7d0d14c71c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9575777fe13344479a4a4948ca3ce945)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:45.850352 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a3e147-f359-4cf6-8587-fd7d0d14c71c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:45.859223 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc7da07-f789-4ecd-9daf-eb0e79bcf9f2 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:45.879062 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a20954-083b-49e9-9975-971502940932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:45.879731 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a20954-083b-49e9-9975-971502940932 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:45.880034 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a20954-083b-49e9-9975-971502940932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:45.880314 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a20954-083b-49e9-9975-971502940932 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:45.881035 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a20954-083b-49e9-9975-971502940932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:45.882813 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a20954-083b-49e9-9975-971502940932 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:45.913057 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 831/1674] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:45 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:45.914273 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57bf901-bd3f-4a8b-9351-e2156d825c7d None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:45.914585 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57bf901-bd3f-4a8b-9351-e2156d825c7d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:45.914866 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57bf901-bd3f-4a8b-9351-e2156d825c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:46.002452 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-c57bf901-bd3f-4a8b-9351-e2156d825c7d None None] MFA Rules not processed for user `5a69202d87094049b12765d64c17e515`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:46.008910 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c57bf901-bd3f-4a8b-9351-e2156d825c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:46.046648 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-e4a3e147-f359-4cf6-8587-fd7d0d14c71c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9575777fe13344479a4a4948ca3ce945', 'request_id': 'req-e4a3e147-f359-4cf6-8587-fd7d0d14c71c'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:46.054543 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:46.073445 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-e4a3e147-f359-4cf6-8587-fd7d0d14c71c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9575777fe13344479a4a4948ca3ce945', 'request_id': 'req-e4a3e147-f359-4cf6-8587-fd7d0d14c71c'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:46.114359 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-e4a3e147-f359-4cf6-8587-fd7d0d14c71c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9575777fe13344479a4a4948ca3ce945', 'request_id': 'req-e4a3e147-f359-4cf6-8587-fd7d0d14c71c'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:46.160211 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-e4a3e147-f359-4cf6-8587-fd7d0d14c71c admin admin] The token cache is being invalidate because project 9575777fe13344479a4a4948ca3ce945 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:21:46.160734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-e4a3e147-f359-4cf6-8587-fd7d0d14c71c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:46.162677 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-e4a3e147-f359-4cf6-8587-fd7d0d14c71c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:46.167734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 844/1675] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:21:45 2022] DELETE /identity/v3/projects/9575777fe13344479a4a4948ca3ce945 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:46.170012 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3e147-f359-4cf6-8587-fd7d0d14c71c admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:46.179081 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e4a3e147-f359-4cf6-8587-fd7d0d14c71c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:46.190193 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 832/1676] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:46.193247 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88d344-6f9c-44d3-b89d-557ac63b25d0 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:46.193865 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88d344-6f9c-44d3-b89d-557ac63b25d0 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:46.194585 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88d344-6f9c-44d3-b89d-557ac63b25d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:46.242486 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-1a88d344-6f9c-44d3-b89d-557ac63b25d0 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:46.247334 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-1a88d344-6f9c-44d3-b89d-557ac63b25d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:46.282316 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:46.296196 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3e147-f359-4cf6-8587-fd7d0d14c71c admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:46.304255 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee66b8-b12b-4d93-af0c-84163eaf7615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:46.305007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee66b8-b12b-4d93-af0c-84163eaf7615 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:46.305674 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee66b8-b12b-4d93-af0c-84163eaf7615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:46.306253 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee66b8-b12b-4d93-af0c-84163eaf7615 admin admin] PATH_INFO: `/v3/projects/92904a77519d4b2f8c5566ec4a2d17c5/users/8dd1d8478f484d2191dbf8188abd4427/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:46.334840 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee66b8-b12b-4d93-af0c-84163eaf7615 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92904a77519d4b2f8c5566ec4a2d17c5, user_id=8dd1d8478f484d2191dbf8188abd4427, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:46.338784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee66b8-b12b-4d93-af0c-84163eaf7615 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:46.366928 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 845/1677] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:21:46 2022] PUT /identity/v3/projects/92904a77519d4b2f8c5566ec4a2d17c5/users/8dd1d8478f484d2191dbf8188abd4427/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:46.368308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee66b8-b12b-4d93-af0c-84163eaf7615 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:46.371733 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-aaee66b8-b12b-4d93-af0c-84163eaf7615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:46.467671 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 833/1678] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:46.469212 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a88d344-6f9c-44d3-b89d-557ac63b25d0 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:46.514619 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a88d344-6f9c-44d3-b89d-557ac63b25d0 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:46.664718 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee66b8-b12b-4d93-af0c-84163eaf7615 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:46.675304 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a138c027-9a57-4fb7-9096-fc61d71e39ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:46.676100 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138c027-9a57-4fb7-9096-fc61d71e39ad admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:46.676386 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138c027-9a57-4fb7-9096-fc61d71e39ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:46.677133 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138c027-9a57-4fb7-9096-fc61d71e39ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:46.677870 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a138c027-9a57-4fb7-9096-fc61d71e39ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:46.679989 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a138c027-9a57-4fb7-9096-fc61d71e39ad admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:46.685921 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a703bc-e85f-43eb-bcaf-f19e689f9f50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:46.686632 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a703bc-e85f-43eb-bcaf-f19e689f9f50 service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:46.686897 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a703bc-e85f-43eb-bcaf-f19e689f9f50 service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:46.687387 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a703bc-e85f-43eb-bcaf-f19e689f9f50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:46.697350 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-04a703bc-e85f-43eb-bcaf-f19e689f9f50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:46.710106 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 834/1679] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:46.710106 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfac32-fb69-4f67-8825-a8a731c7961f None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:46.710106 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfac32-fb69-4f67-8825-a8a731c7961f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:46.710496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfac32-fb69-4f67-8825-a8a731c7961f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:46.781258 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-c5dfac32-fb69-4f67-8825-a8a731c7961f None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:46.782540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c5dfac32-fb69-4f67-8825-a8a731c7961f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:46.806412 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 835/1680] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:46.810236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfac32-fb69-4f67-8825-a8a731c7961f None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:46.835366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfac32-fb69-4f67-8825-a8a731c7961f None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:46.848730 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e7a76-b95e-4930-a63c-716f131e26d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:46.849444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e7a76-b95e-4930-a63c-716f131e26d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:46.849728 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e7a76-b95e-4930-a63c-716f131e26d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:46.850023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e7a76-b95e-4930-a63c-716f131e26d9 admin admin] PATH_INFO: `/v3/projects/92904a77519d4b2f8c5566ec4a2d17c5/users/8dd1d8478f484d2191dbf8188abd4427/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:46.855249 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e7a76-b95e-4930-a63c-716f131e26d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92904a77519d4b2f8c5566ec4a2d17c5, user_id=8dd1d8478f484d2191dbf8188abd4427, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:46.863891 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e7a76-b95e-4930-a63c-716f131e26d9 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:46.890630 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 836/1681] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:21:46 2022] PUT /identity/v3/projects/92904a77519d4b2f8c5566ec4a2d17c5/users/8dd1d8478f484d2191dbf8188abd4427/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:46.891262 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e7a76-b95e-4930-a63c-716f131e26d9 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:46.916921 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a703bc-e85f-43eb-bcaf-f19e689f9f50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:46.918580 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a703bc-e85f-43eb-bcaf-f19e689f9f50 service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:46.963057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 846/1682] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:21:46 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:21:46.965193 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e9088-59e8-43d8-addd-250a21cf077c None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:46.965902 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e9088-59e8-43d8-addd-250a21cf077c None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:46.965902 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e9088-59e8-43d8-addd-250a21cf077c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:47.015666 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-398e9088-59e8-43d8-addd-250a21cf077c None None] MFA Rules not processed for user `8dd1d8478f484d2191dbf8188abd4427`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:47.017129 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-398e9088-59e8-43d8-addd-250a21cf077c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:47.054680 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:47.098982 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e7a76-b95e-4930-a63c-716f131e26d9 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:47.112773 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f006e47-77ca-44b1-8d8e-14352f122441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:47.114158 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f006e47-77ca-44b1-8d8e-14352f122441 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:47.114158 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f006e47-77ca-44b1-8d8e-14352f122441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:47.114618 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f006e47-77ca-44b1-8d8e-14352f122441 admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:47.115460 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f006e47-77ca-44b1-8d8e-14352f122441 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:21:47.115877 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f006e47-77ca-44b1-8d8e-14352f122441 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:47.120657 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f006e47-77ca-44b1-8d8e-14352f122441 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:47.157918 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 837/1683] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:21:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:21:47.159689 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f006e47-77ca-44b1-8d8e-14352f122441 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:47.185821 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f006e47-77ca-44b1-8d8e-14352f122441 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:47.195476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:47.200612 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733eb560-4a4d-4953-950f-3338c24563de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:47.202740 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733eb560-4a4d-4953-950f-3338c24563de admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:47.203141 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733eb560-4a4d-4953-950f-3338c24563de admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:47.204507 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733eb560-4a4d-4953-950f-3338c24563de admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:47.205385 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733eb560-4a4d-4953-950f-3338c24563de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:47.207810 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 847/1684] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:46 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:47.209544 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49559718-97fe-4c10-8dbe-9b3e9d619766 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:47.210101 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49559718-97fe-4c10-8dbe-9b3e9d619766 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:47.210436 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733eb560-4a4d-4953-950f-3338c24563de admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:47.210977 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49559718-97fe-4c10-8dbe-9b3e9d619766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:47.301546 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 838/1685] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:21:47 2022] POST /identity/v3/projects => generated 363 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:47.302743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733eb560-4a4d-4953-950f-3338c24563de admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:47.305647 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-49559718-97fe-4c10-8dbe-9b3e9d619766 None None] MFA Rules not processed for user `a430d7aa4dcf40e399fcb5d1f4bf3f76`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:47.307135 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-49559718-97fe-4c10-8dbe-9b3e9d619766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:47.310789 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-733eb560-4a4d-4953-950f-3338c24563de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:47.480763 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 848/1686] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:47.480763 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49559718-97fe-4c10-8dbe-9b3e9d619766 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:47.513844 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733eb560-4a4d-4953-950f-3338c24563de admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:47.538916 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cee35b-e04b-452f-9c1a-acbfe4dc0cc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:47.540110 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cee35b-e04b-452f-9c1a-acbfe4dc0cc8 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:47.540564 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cee35b-e04b-452f-9c1a-acbfe4dc0cc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:47.540846 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cee35b-e04b-452f-9c1a-acbfe4dc0cc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:47.550879 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-79cee35b-e04b-452f-9c1a-acbfe4dc0cc8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:47.636036 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49559718-97fe-4c10-8dbe-9b3e9d619766 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:47.646818 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a5df7-ac76-4888-8c4b-2c1ac02f3319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:47.647695 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a5df7-ac76-4888-8c4b-2c1ac02f3319 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:47.648316 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a5df7-ac76-4888-8c4b-2c1ac02f3319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:47.648590 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a5df7-ac76-4888-8c4b-2c1ac02f3319 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:47.649393 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a5df7-ac76-4888-8c4b-2c1ac02f3319 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:47.654728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a5df7-ac76-4888-8c4b-2c1ac02f3319 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:47.692555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 849/1687] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:21:47 2022] POST /identity/v3/users => generated 282 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:47.694802 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40965627-b9c2-4970-b545-8b96fd254ad8 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:47.695230 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40965627-b9c2-4970-b545-8b96fd254ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:47.695760 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40965627-b9c2-4970-b545-8b96fd254ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:47.720045 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-40965627-b9c2-4970-b545-8b96fd254ad8 None None] MFA Rules not processed for user `a430d7aa4dcf40e399fcb5d1f4bf3f76`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:47.726273 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-40965627-b9c2-4970-b545-8b96fd254ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:47.732363 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 850/1688] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:47.733916 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d402c0-b6f1-4274-a9fb-798815e91b50 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:47.734468 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d402c0-b6f1-4274-a9fb-798815e91b50 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:47.734874 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d402c0-b6f1-4274-a9fb-798815e91b50 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:47.737232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 851/1689] 10.222.0.46 () {54 vars in 921 bytes} [Wed Jul 27 04:21:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:21:47.738579 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d402c0-b6f1-4274-a9fb-798815e91b50 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:47.755917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d402c0-b6f1-4274-a9fb-798815e91b50 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:47.763567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98762f25-c80a-4e82-b491-ef7c632c8d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:47.764186 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98762f25-c80a-4e82-b491-ef7c632c8d73 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:47.764938 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98762f25-c80a-4e82-b491-ef7c632c8d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:47.765379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98762f25-c80a-4e82-b491-ef7c632c8d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:47.766551 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98762f25-c80a-4e82-b491-ef7c632c8d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:47.768728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98762f25-c80a-4e82-b491-ef7c632c8d73 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:47.786449 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 852/1690] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:47.788675 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006afd1c-b78c-4776-beca-678abebf682f None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:47.789175 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006afd1c-b78c-4776-beca-678abebf682f None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:47.789897 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006afd1c-b78c-4776-beca-678abebf682f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:47.818219 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-006afd1c-b78c-4776-beca-678abebf682f None None] MFA Rules not processed for user `8c08fba2d7de42218400c5e652cc3de0`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:47.820016 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-006afd1c-b78c-4776-beca-678abebf682f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:47.820667 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cee35b-e04b-452f-9c1a-acbfe4dc0cc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:47.833067 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cee35b-e04b-452f-9c1a-acbfe4dc0cc8 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:47.867968 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 839/1691] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:21:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:21:47.869450 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cee35b-e04b-452f-9c1a-acbfe4dc0cc8 service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:47.907700 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cee35b-e04b-452f-9c1a-acbfe4dc0cc8 service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:47.919053 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f736352-c684-4e8c-a260-ced4585d32f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:47.920048 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f736352-c684-4e8c-a260-ced4585d32f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:47.920534 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f736352-c684-4e8c-a260-ced4585d32f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:47.920880 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f736352-c684-4e8c-a260-ced4585d32f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:47.922210 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f736352-c684-4e8c-a260-ced4585d32f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:47.924189 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f736352-c684-4e8c-a260-ced4585d32f3 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:47.952559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 840/1692] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:47.960495 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f736352-c684-4e8c-a260-ced4585d32f3 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:47.974114 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:47.997398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f736352-c684-4e8c-a260-ced4585d32f3 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:47.999104 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 853/1693] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Jul 27 04:21:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:48.011858 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a362464a-dd89-473b-84aa-3ce6229966b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:48.012789 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362464a-dd89-473b-84aa-3ce6229966b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:48.013091 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362464a-dd89-473b-84aa-3ce6229966b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:48.013345 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362464a-dd89-473b-84aa-3ce6229966b1 admin admin] PATH_INFO: `/v3/projects/1e375d85803c459c8905bdcd5a0c6b67/users/f55e390244af4583aea3b0b36bbf3ebb/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:48.053598 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71733e9-9054-4b2a-bb6d-00e1ebbdc19c None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:48.053889 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71733e9-9054-4b2a-bb6d-00e1ebbdc19c None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:48.054249 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71733e9-9054-4b2a-bb6d-00e1ebbdc19c None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:48.056157 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 854/1694] 10.222.0.46 () {54 vars in 961 bytes} [Wed Jul 27 04:21:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:21:48.067274 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0535190-2c7d-4d8d-a349-fea18bb924b9 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:48.067667 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0535190-2c7d-4d8d-a349-fea18bb924b9 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:48.067934 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0535190-2c7d-4d8d-a349-fea18bb924b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:48.074328 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a362464a-dd89-473b-84aa-3ce6229966b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e375d85803c459c8905bdcd5a0c6b67, user_id=f55e390244af4583aea3b0b36bbf3ebb, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:48.081478 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a362464a-dd89-473b-84aa-3ce6229966b1 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:48.115894 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-b0535190-2c7d-4d8d-a349-fea18bb924b9 None None] MFA Rules not processed for user `8c08fba2d7de42218400c5e652cc3de0`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:48.118676 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b0535190-2c7d-4d8d-a349-fea18bb924b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:48.132963 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 841/1695] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:21:47 2022] PUT /identity/v3/projects/1e375d85803c459c8905bdcd5a0c6b67/users/f55e390244af4583aea3b0b36bbf3ebb/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:48.139486 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a362464a-dd89-473b-84aa-3ce6229966b1 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:48.145132 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-a362464a-dd89-473b-84aa-3ce6229966b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:48.239168 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:48.338399 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 855/1696] 10.222.0.46 () {58 vars in 1058 bytes} [Wed Jul 27 04:21:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:21:48.340023 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e00b249-8c43-4ae3-a652-21e79866a174 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:48.340592 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e00b249-8c43-4ae3-a652-21e79866a174 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:48.340806 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e00b249-8c43-4ae3-a652-21e79866a174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:48.350048 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a362464a-dd89-473b-84aa-3ce6229966b1 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:48.365282 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e3fa9-d709-4bdd-a1e9-c75a626c221c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:48.366775 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e3fa9-d709-4bdd-a1e9-c75a626c221c service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:48.367066 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e3fa9-d709-4bdd-a1e9-c75a626c221c service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:48.367348 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e3fa9-d709-4bdd-a1e9-c75a626c221c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:48.382972 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-8e00b249-8c43-4ae3-a652-21e79866a174 None None] MFA Rules not processed for user `f55e390244af4583aea3b0b36bbf3ebb`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:48.386733 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8e00b249-8c43-4ae3-a652-21e79866a174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:48.530603 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517e3fa9-d709-4bdd-a1e9-c75a626c221c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:48.534215 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517e3fa9-d709-4bdd-a1e9-c75a626c221c service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:48.562539 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 842/1697] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:21:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:21:48.563985 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d0d7e-576e-47c7-b631-268acb8ffa4a None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:48.564275 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d0d7e-576e-47c7-b631-268acb8ffa4a None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:48.564551 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d0d7e-576e-47c7-b631-268acb8ffa4a None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:48.565892 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 843/1698] 10.222.0.46 () {54 vars in 912 bytes} [Wed Jul 27 04:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:21:48.582656 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d0d7e-576e-47c7-b631-268acb8ffa4a None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:48.622703 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d0d7e-576e-47c7-b631-268acb8ffa4a None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:48.631365 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c3c79-8fcc-4458-825c-96acf50c7591 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c08fba2d7de42218400c5e652cc3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:48.632298 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c3c79-8fcc-4458-825c-96acf50c7591 service placement] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:48.632784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c3c79-8fcc-4458-825c-96acf50c7591 service placement] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:48.633258 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c3c79-8fcc-4458-825c-96acf50c7591 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:48.655546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 856/1699] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:48 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:48.659010 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6af47-fbfc-42f7-8c21-40a6c2b1201e None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:48.659306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6af47-fbfc-42f7-8c21-40a6c2b1201e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:48.659584 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6af47-fbfc-42f7-8c21-40a6c2b1201e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:48.664098 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09c3c79-8fcc-4458-825c-96acf50c7591 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:48.667234 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09c3c79-8fcc-4458-825c-96acf50c7591 service placement] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:48.711559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 844/1700] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:21:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:21:48.721941 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c3c79-8fcc-4458-825c-96acf50c7591 service placement] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:48.759573 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-e1e6af47-fbfc-42f7-8c21-40a6c2b1201e None None] MFA Rules not processed for user `b4847f3736d844d78a206b7eab5f5fcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:48.760986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e1e6af47-fbfc-42f7-8c21-40a6c2b1201e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:48.897255 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c3c79-8fcc-4458-825c-96acf50c7591 service placement] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:48.900404 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 857/1701] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Jul 27 04:21:48 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:48.905158 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f071b2-d9a6-4fee-b77e-f22d9be23ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:48.905844 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f071b2-d9a6-4fee-b77e-f22d9be23ecb admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:48.906103 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f071b2-d9a6-4fee-b77e-f22d9be23ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:48.906379 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f071b2-d9a6-4fee-b77e-f22d9be23ecb admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:48.907060 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f071b2-d9a6-4fee-b77e-f22d9be23ecb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:48.909456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f071b2-d9a6-4fee-b77e-f22d9be23ecb admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:48.917241 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e6af47-fbfc-42f7-8c21-40a6c2b1201e None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:48.919238 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e1e6af47-fbfc-42f7-8c21-40a6c2b1201e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:48.941281 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e6af47-fbfc-42f7-8c21-40a6c2b1201e None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:48.950529 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e618d4-62ef-4054-b197-1519a4c3dd88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:48.951384 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e618d4-62ef-4054-b197-1519a4c3dd88 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:48.951665 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e618d4-62ef-4054-b197-1519a4c3dd88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:48.951938 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e618d4-62ef-4054-b197-1519a4c3dd88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:48.964375 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e618d4-62ef-4054-b197-1519a4c3dd88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:48.965919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e618d4-62ef-4054-b197-1519a4c3dd88 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:48.983469 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 845/1702] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:21:48 2022] POST /identity/v3/projects => generated 363 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:48.984425 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f071b2-d9a6-4fee-b77e-f22d9be23ecb admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:48.999992 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 858/1703] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:21:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 04:21:49.073889 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:49.122422 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f071b2-d9a6-4fee-b77e-f22d9be23ecb admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:49.142293 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe95360-3da9-4349-adae-f490e739635e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:49.142961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe95360-3da9-4349-adae-f490e739635e admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:49.143239 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe95360-3da9-4349-adae-f490e739635e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:49.143511 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe95360-3da9-4349-adae-f490e739635e admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:49.144198 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe95360-3da9-4349-adae-f490e739635e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:49.146194 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe95360-3da9-4349-adae-f490e739635e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:49.198418 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 846/1704] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:21:48 2022] POST /identity/v3/users => generated 288 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:49.203131 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e618d4-62ef-4054-b197-1519a4c3dd88 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:49.216475 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e618d4-62ef-4054-b197-1519a4c3dd88 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:49.226155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2dcbcd-8ce0-4e18-a5cd-6f7c588e16fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:49.226881 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2dcbcd-8ce0-4e18-a5cd-6f7c588e16fd admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:49.227168 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2dcbcd-8ce0-4e18-a5cd-6f7c588e16fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:49.227442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2dcbcd-8ce0-4e18-a5cd-6f7c588e16fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:49.230340 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2dcbcd-8ce0-4e18-a5cd-6f7c588e16fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:49.234754 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2dcbcd-8ce0-4e18-a5cd-6f7c588e16fd admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:49.239090 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:49.269922 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe95360-3da9-4349-adae-f490e739635e admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:49.281127 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 859/1705] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:49.288649 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe95360-3da9-4349-adae-f490e739635e admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:49.300186 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ac33a-00e7-4d65-9edc-5a2260a33028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:49.301068 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ac33a-00e7-4d65-9edc-5a2260a33028 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:49.301497 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ac33a-00e7-4d65-9edc-5a2260a33028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:49.301883 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ac33a-00e7-4d65-9edc-5a2260a33028 admin admin] PATH_INFO: `/v3/projects/89c108cc7727445394cf435943a6a0b6/users/ec19e8c7484241489edaa2c23f62a971/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:49.345873 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ac33a-00e7-4d65-9edc-5a2260a33028 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89c108cc7727445394cf435943a6a0b6, user_id=ec19e8c7484241489edaa2c23f62a971, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:49.348761 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ac33a-00e7-4d65-9edc-5a2260a33028 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:49.374571 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 847/1706] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:21:49 2022] PUT /identity/v3/projects/89c108cc7727445394cf435943a6a0b6/users/ec19e8c7484241489edaa2c23f62a971/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:21:49.379135 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2dcbcd-8ce0-4e18-a5cd-6f7c588e16fd admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:49.413952 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2dcbcd-8ce0-4e18-a5cd-6f7c588e16fd admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:49.618591 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447a116-678b-4784-adb7-8a04c4935abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:49.620246 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447a116-678b-4784-adb7-8a04c4935abc admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:49.620724 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447a116-678b-4784-adb7-8a04c4935abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:49.623654 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447a116-678b-4784-adb7-8a04c4935abc admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:49.625711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9447a116-678b-4784-adb7-8a04c4935abc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:49.631841 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9447a116-678b-4784-adb7-8a04c4935abc admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:49.664614 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 860/1707] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:49.666936 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ac33a-00e7-4d65-9edc-5a2260a33028 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:49.673934 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42787100-999a-4638-bb84-495db6c02c72 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:49.674243 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42787100-999a-4638-bb84-495db6c02c72 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:49.674521 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42787100-999a-4638-bb84-495db6c02c72 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:49.697222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 861/1708] 10.222.0.46 () {54 vars in 912 bytes} [Wed Jul 27 04:21:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:21:49.701816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ac33a-00e7-4d65-9edc-5a2260a33028 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:49.704939 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03533194-e1b1-4d9c-ba58-41c99f7b02bf None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:49.705228 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03533194-e1b1-4d9c-ba58-41c99f7b02bf None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:49.705510 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03533194-e1b1-4d9c-ba58-41c99f7b02bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:49.712328 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b7cfcf-fcc0-4389-8498-afba045d1a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:49.713055 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b7cfcf-fcc0-4389-8498-afba045d1a6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:49.713364 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b7cfcf-fcc0-4389-8498-afba045d1a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:49.714059 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b7cfcf-fcc0-4389-8498-afba045d1a6f admin admin] PATH_INFO: `/v3/projects/89c108cc7727445394cf435943a6a0b6/users/ec19e8c7484241489edaa2c23f62a971/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:49.719634 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b7cfcf-fcc0-4389-8498-afba045d1a6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=89c108cc7727445394cf435943a6a0b6, user_id=ec19e8c7484241489edaa2c23f62a971, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:49.728535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b7cfcf-fcc0-4389-8498-afba045d1a6f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:49.754983 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 848/1709] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:21:49 2022] PUT /identity/v3/projects/89c108cc7727445394cf435943a6a0b6/users/ec19e8c7484241489edaa2c23f62a971/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:49.769454 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-03533194-e1b1-4d9c-ba58-41c99f7b02bf None None] MFA Rules not processed for user `b4847f3736d844d78a206b7eab5f5fcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:49.770042 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf113173-931f-4568-899e-b364601e999c None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:49.770349 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf113173-931f-4568-899e-b364601e999c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:49.770633 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf113173-931f-4568-899e-b364601e999c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:49.771838 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-03533194-e1b1-4d9c-ba58-41c99f7b02bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:49.843836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-bf113173-931f-4568-899e-b364601e999c None None] MFA Rules not processed for user `ec19e8c7484241489edaa2c23f62a971`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:49.845037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-bf113173-931f-4568-899e-b364601e999c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:49.896654 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 862/1710] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Jul 27 04:21:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:49.898198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3695d30e-6cf6-4d75-9e95-70474a0c75ad None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:49.898495 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3695d30e-6cf6-4d75-9e95-70474a0c75ad None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:49.898844 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3695d30e-6cf6-4d75-9e95-70474a0c75ad None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:49.901282 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 863/1711] 10.222.0.46 () {54 vars in 921 bytes} [Wed Jul 27 04:21:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:21:49.927265 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3695d30e-6cf6-4d75-9e95-70474a0c75ad None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:49.927265 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-3695d30e-6cf6-4d75-9e95-70474a0c75ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:49.976481 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3695d30e-6cf6-4d75-9e95-70474a0c75ad None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:49.984242 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd4bb6-db68-4aab-a98c-5a789d7fbc15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:49.985019 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd4bb6-db68-4aab-a98c-5a789d7fbc15 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:49.985412 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd4bb6-db68-4aab-a98c-5a789d7fbc15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:49.988193 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd4bb6-db68-4aab-a98c-5a789d7fbc15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:50.003422 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd4bb6-db68-4aab-a98c-5a789d7fbc15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:50.010981 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd4bb6-db68-4aab-a98c-5a789d7fbc15 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:50.023287 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 864/1712] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:21:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:21:50.025449 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd4bb6-db68-4aab-a98c-5a789d7fbc15 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:50.027459 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d2cd4bb6-db68-4aab-a98c-5a789d7fbc15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:50.095433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 849/1713] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:49 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:50.106173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1276b-a7d4-4cfb-81c7-e327914c330b None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:50.106871 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1276b-a7d4-4cfb-81c7-e327914c330b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:50.106871 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1276b-a7d4-4cfb-81c7-e327914c330b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:50.144274 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-5fa1276b-a7d4-4cfb-81c7-e327914c330b None None] MFA Rules not processed for user `8c08fba2d7de42218400c5e652cc3de0`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:50.148711 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-5fa1276b-a7d4-4cfb-81c7-e327914c330b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:50.257011 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd4bb6-db68-4aab-a98c-5a789d7fbc15 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:50.274419 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9041c2-5a2d-461c-95b8-fa014127f77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:50.275110 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9041c2-5a2d-461c-95b8-fa014127f77f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:50.275414 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9041c2-5a2d-461c-95b8-fa014127f77f admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:50.276249 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9041c2-5a2d-461c-95b8-fa014127f77f admin admin] PATH_INFO: `/v3/users/97602bd0c5ce4e11819b634995d93405` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:50.277571 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9041c2-5a2d-461c-95b8-fa014127f77f admin admin] RBAC: Authorizing `identity:delete_user(user_id=97602bd0c5ce4e11819b634995d93405)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:50.280310 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9041c2-5a2d-461c-95b8-fa014127f77f admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:50.292611 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 850/1714] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Jul 27 04:21:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:50.293850 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1276b-a7d4-4cfb-81c7-e327914c330b None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:50.360862 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:50.364276 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:50.398706 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7a9041c2-5a2d-461c-95b8-fa014127f77f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97602bd0c5ce4e11819b634995d93405'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:50.403001 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7a9041c2-5a2d-461c-95b8-fa014127f77f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97602bd0c5ce4e11819b634995d93405'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:50.404151 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7a9041c2-5a2d-461c-95b8-fa014127f77f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97602bd0c5ce4e11819b634995d93405'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:50.436246 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7a9041c2-5a2d-461c-95b8-fa014127f77f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97602bd0c5ce4e11819b634995d93405'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:50.448135 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7a9041c2-5a2d-461c-95b8-fa014127f77f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97602bd0c5ce4e11819b634995d93405'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:50.451700 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1276b-a7d4-4cfb-81c7-e327914c330b None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:50.466816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34598ba3-00e8-45bf-ba4d-da21ec3d3f35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:50.466816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34598ba3-00e8-45bf-ba4d-da21ec3d3f35 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:50.466816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34598ba3-00e8-45bf-ba4d-da21ec3d3f35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:50.466816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34598ba3-00e8-45bf-ba4d-da21ec3d3f35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:50.468617 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-34598ba3-00e8-45bf-ba4d-da21ec3d3f35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:50.479599 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 865/1715] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:21:50 2022] DELETE /identity/v3/users/97602bd0c5ce4e11819b634995d93405 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:50.481179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9041c2-5a2d-461c-95b8-fa014127f77f admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:50.485810 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7a9041c2-5a2d-461c-95b8-fa014127f77f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:50.508367 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34598ba3-00e8-45bf-ba4d-da21ec3d3f35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:50.509965 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34598ba3-00e8-45bf-ba4d-da21ec3d3f35 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:50.554016 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 851/1716] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:21:50 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:21:50.654907 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34598ba3-00e8-45bf-ba4d-da21ec3d3f35 service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:50.667915 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-34598ba3-00e8-45bf-ba4d-da21ec3d3f35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:50.679845 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9041c2-5a2d-461c-95b8-fa014127f77f admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:50.689418 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ba9e0-9955-46d3-820f-be55595d2d3c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c08fba2d7de42218400c5e652cc3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:50.693018 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ba9e0-9955-46d3-820f-be55595d2d3c service placement] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:50.695094 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ba9e0-9955-46d3-820f-be55595d2d3c service placement] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:50.696955 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ba9e0-9955-46d3-820f-be55595d2d3c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:50.703827 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-905ba9e0-9955-46d3-820f-be55595d2d3c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:50.709268 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34598ba3-00e8-45bf-ba4d-da21ec3d3f35 service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:50.722821 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d91d2-3aa1-4925-904b-0624036918aa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c08fba2d7de42218400c5e652cc3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:50.728084 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d91d2-3aa1-4925-904b-0624036918aa service placement] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:50.728234 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905ba9e0-9955-46d3-820f-be55595d2d3c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:50.729247 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d91d2-3aa1-4925-904b-0624036918aa service placement] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:50.729901 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d91d2-3aa1-4925-904b-0624036918aa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:50.731639 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905ba9e0-9955-46d3-820f-be55595d2d3c service placement] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:50.735693 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-727d91d2-3aa1-4925-904b-0624036918aa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:50.767067 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 866/1717] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:21:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:21:50.774328 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d91d2-3aa1-4925-904b-0624036918aa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:50.777728 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d91d2-3aa1-4925-904b-0624036918aa service placement] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:50.858734 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 852/1718] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:21:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:21:50.975723 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ba9e0-9955-46d3-820f-be55595d2d3c service placement] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:50.977623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-905ba9e0-9955-46d3-820f-be55595d2d3c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:51.111698 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ba9e0-9955-46d3-820f-be55595d2d3c service placement] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:51.118383 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3848599-3f54-4a6e-bfb6-05710b54ef48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:51.120659 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3848599-3f54-4a6e-bfb6-05710b54ef48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:51.121142 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3848599-3f54-4a6e-bfb6-05710b54ef48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:51.121521 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3848599-3f54-4a6e-bfb6-05710b54ef48 admin admin] PATH_INFO: `/v3/projects/904ec432f16b407ea0dd54a3caf637b4` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:51.122941 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3848599-3f54-4a6e-bfb6-05710b54ef48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=904ec432f16b407ea0dd54a3caf637b4)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:51.125273 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3848599-3f54-4a6e-bfb6-05710b54ef48 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:51.212945 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:51.229504 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c3848599-3f54-4a6e-bfb6-05710b54ef48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '904ec432f16b407ea0dd54a3caf637b4', 'request_id': 'req-c3848599-3f54-4a6e-bfb6-05710b54ef48'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:51.240769 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c3848599-3f54-4a6e-bfb6-05710b54ef48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '904ec432f16b407ea0dd54a3caf637b4', 'request_id': 'req-c3848599-3f54-4a6e-bfb6-05710b54ef48'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:51.252478 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c3848599-3f54-4a6e-bfb6-05710b54ef48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '904ec432f16b407ea0dd54a3caf637b4', 'request_id': 'req-c3848599-3f54-4a6e-bfb6-05710b54ef48'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:51.283052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c3848599-3f54-4a6e-bfb6-05710b54ef48 admin admin] The token cache is being invalidate because project 904ec432f16b407ea0dd54a3caf637b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:21:51.283531 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c3848599-3f54-4a6e-bfb6-05710b54ef48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:51.285496 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c3848599-3f54-4a6e-bfb6-05710b54ef48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:51.289678 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 867/1719] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:21:50 2022] DELETE /identity/v3/projects/904ec432f16b407ea0dd54a3caf637b4 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:51.293863 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb150e57-6f0d-4e7b-9fa7-d41ec5a1089d None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:51.294367 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3848599-3f54-4a6e-bfb6-05710b54ef48 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:51.294441 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb150e57-6f0d-4e7b-9fa7-d41ec5a1089d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:51.294507 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb150e57-6f0d-4e7b-9fa7-d41ec5a1089d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:51.295449 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c3848599-3f54-4a6e-bfb6-05710b54ef48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:51.324233 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-fb150e57-6f0d-4e7b-9fa7-d41ec5a1089d None None] MFA Rules not processed for user `5a69202d87094049b12765d64c17e515`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:51.325458 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-fb150e57-6f0d-4e7b-9fa7-d41ec5a1089d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:51.542716 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 853/1720] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:51.560181 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3848599-3f54-4a6e-bfb6-05710b54ef48 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:51.589889 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeda881a-318b-4f87-8dd5-0588e3a7ca61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:51.605613 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda881a-318b-4f87-8dd5-0588e3a7ca61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:51.605880 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda881a-318b-4f87-8dd5-0588e3a7ca61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:51.606198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda881a-318b-4f87-8dd5-0588e3a7ca61 admin admin] PATH_INFO: `/v3/users/c2cd62a35ffb4338a6387ce793622407` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:51.611303 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeda881a-318b-4f87-8dd5-0588e3a7ca61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2cd62a35ffb4338a6387ce793622407)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:51.613328 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeda881a-318b-4f87-8dd5-0588e3a7ca61 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:51.690052 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb150e57-6f0d-4e7b-9fa7-d41ec5a1089d None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:51.692970 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-fb150e57-6f0d-4e7b-9fa7-d41ec5a1089d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:51.748452 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb150e57-6f0d-4e7b-9fa7-d41ec5a1089d None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:51.764433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924bd6c9-ef06-4dac-986b-572d82dbe30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:51.765107 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924bd6c9-ef06-4dac-986b-572d82dbe30e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:51.765391 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924bd6c9-ef06-4dac-986b-572d82dbe30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:51.765701 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924bd6c9-ef06-4dac-986b-572d82dbe30e admin admin] PATH_INFO: `/v3/users/7845ef787bc5446786f5c7ce9dea9863` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:51.790209 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-aeda881a-318b-4f87-8dd5-0588e3a7ca61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2cd62a35ffb4338a6387ce793622407'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:51.790584 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924bd6c9-ef06-4dac-986b-572d82dbe30e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7845ef787bc5446786f5c7ce9dea9863)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:51.790959 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-aeda881a-318b-4f87-8dd5-0588e3a7ca61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2cd62a35ffb4338a6387ce793622407'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:51.792170 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-aeda881a-318b-4f87-8dd5-0588e3a7ca61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2cd62a35ffb4338a6387ce793622407'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:51.794107 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924bd6c9-ef06-4dac-986b-572d82dbe30e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:51.826860 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-aeda881a-318b-4f87-8dd5-0588e3a7ca61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2cd62a35ffb4338a6387ce793622407'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:51.850298 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-aeda881a-318b-4f87-8dd5-0588e3a7ca61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2cd62a35ffb4338a6387ce793622407'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:51.887612 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 868/1721] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:21:51 2022] DELETE /identity/v3/users/c2cd62a35ffb4338a6387ce793622407 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:51.952190 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-924bd6c9-ef06-4dac-986b-572d82dbe30e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7845ef787bc5446786f5c7ce9dea9863'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:51.953453 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-924bd6c9-ef06-4dac-986b-572d82dbe30e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7845ef787bc5446786f5c7ce9dea9863'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:51.972109 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-924bd6c9-ef06-4dac-986b-572d82dbe30e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7845ef787bc5446786f5c7ce9dea9863'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:52.023993 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-924bd6c9-ef06-4dac-986b-572d82dbe30e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7845ef787bc5446786f5c7ce9dea9863'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:52.049389 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-924bd6c9-ef06-4dac-986b-572d82dbe30e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7845ef787bc5446786f5c7ce9dea9863'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:52.071174 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 854/1722] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:21:51 2022] DELETE /identity/v3/users/7845ef787bc5446786f5c7ce9dea9863 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:21:52.074606 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3cf5d-6204-4abc-85d8-2026e0a182fa None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:52.075060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3cf5d-6204-4abc-85d8-2026e0a182fa None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:52.075622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3cf5d-6204-4abc-85d8-2026e0a182fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:52.123404 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-f5d3cf5d-6204-4abc-85d8-2026e0a182fa None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:52.125455 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f5d3cf5d-6204-4abc-85d8-2026e0a182fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:52.158220 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924bd6c9-ef06-4dac-986b-572d82dbe30e admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:52.160111 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-924bd6c9-ef06-4dac-986b-572d82dbe30e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:52.236720 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:52.321774 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:52.372684 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 869/1723] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:52.390123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3cf5d-6204-4abc-85d8-2026e0a182fa None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:52.391880 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f5d3cf5d-6204-4abc-85d8-2026e0a182fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:52.394478 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924bd6c9-ef06-4dac-986b-572d82dbe30e admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:52.407655 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46709e4-59e8-4c9b-be12-138cf665d2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:52.409714 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46709e4-59e8-4c9b-be12-138cf665d2aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:52.410230 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46709e4-59e8-4c9b-be12-138cf665d2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:52.410673 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46709e4-59e8-4c9b-be12-138cf665d2aa admin admin] PATH_INFO: `/v3/users/f55e390244af4583aea3b0b36bbf3ebb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:52.412723 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46709e4-59e8-4c9b-be12-138cf665d2aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=f55e390244af4583aea3b0b36bbf3ebb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:52.434897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46709e4-59e8-4c9b-be12-138cf665d2aa admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:52.567847 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f46709e4-59e8-4c9b-be12-138cf665d2aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f55e390244af4583aea3b0b36bbf3ebb'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:52.567847 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f46709e4-59e8-4c9b-be12-138cf665d2aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f55e390244af4583aea3b0b36bbf3ebb'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:52.590663 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f46709e4-59e8-4c9b-be12-138cf665d2aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f55e390244af4583aea3b0b36bbf3ebb'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:52.615375 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3cf5d-6204-4abc-85d8-2026e0a182fa None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:52.626356 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99479c91-6b78-4057-b4a2-4ebf0cb47b36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:52.627131 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99479c91-6b78-4057-b4a2-4ebf0cb47b36 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:52.627495 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99479c91-6b78-4057-b4a2-4ebf0cb47b36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:52.627936 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99479c91-6b78-4057-b4a2-4ebf0cb47b36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:52.635765 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f46709e4-59e8-4c9b-be12-138cf665d2aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f55e390244af4583aea3b0b36bbf3ebb'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:52.652357 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f46709e4-59e8-4c9b-be12-138cf665d2aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f55e390244af4583aea3b0b36bbf3ebb'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:52.655827 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99479c91-6b78-4057-b4a2-4ebf0cb47b36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:52.662399 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 855/1724] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:21:52 2022] DELETE /identity/v3/users/f55e390244af4583aea3b0b36bbf3ebb => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:52.663137 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99479c91-6b78-4057-b4a2-4ebf0cb47b36 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:52.663245 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46709e4-59e8-4c9b-be12-138cf665d2aa admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:52.672892 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f46709e4-59e8-4c9b-be12-138cf665d2aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:52.690036 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 870/1725] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:21:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 04:21:52.691294 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27c11b-6da5-4983-bab0-94f785384c96 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:52.691584 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27c11b-6da5-4983-bab0-94f785384c96 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:52.691862 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27c11b-6da5-4983-bab0-94f785384c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:52.754726 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-ac27c11b-6da5-4983-bab0-94f785384c96 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:52.756285 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-ac27c11b-6da5-4983-bab0-94f785384c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:52.857836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46709e4-59e8-4c9b-be12-138cf665d2aa admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:52.873128 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb0e24-c894-4a60-9b9f-8e6751fe1ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:52.873955 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb0e24-c894-4a60-9b9f-8e6751fe1ec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:52.874363 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb0e24-c894-4a60-9b9f-8e6751fe1ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:52.875302 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb0e24-c894-4a60-9b9f-8e6751fe1ec8 admin admin] PATH_INFO: `/v3/projects/d9f6e6209ad24963b27eb9a6aabbe092` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:52.877008 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb0e24-c894-4a60-9b9f-8e6751fe1ec8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9f6e6209ad24963b27eb9a6aabbe092)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:52.878214 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: WARNING py.warnings [None req-a9cb0e24-c894-4a60-9b9f-8e6751fe1ec8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 04:21:52.878214 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: warnings.warn(msg) Jul 27 04:21:52.878214 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: Jul 27 04:21:52.882767 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb0e24-c894-4a60-9b9f-8e6751fe1ec8 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:52.892159 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 871/1726] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:52.909997 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27c11b-6da5-4983-bab0-94f785384c96 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:52.911479 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-ac27c11b-6da5-4983-bab0-94f785384c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:53.000499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:53.017879 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27c11b-6da5-4983-bab0-94f785384c96 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:53.020098 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a9cb0e24-c894-4a60-9b9f-8e6751fe1ec8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9f6e6209ad24963b27eb9a6aabbe092', 'request_id': 'req-a9cb0e24-c894-4a60-9b9f-8e6751fe1ec8'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:53.047127 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a9cb0e24-c894-4a60-9b9f-8e6751fe1ec8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9f6e6209ad24963b27eb9a6aabbe092', 'request_id': 'req-a9cb0e24-c894-4a60-9b9f-8e6751fe1ec8'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:53.059239 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a9cb0e24-c894-4a60-9b9f-8e6751fe1ec8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9f6e6209ad24963b27eb9a6aabbe092', 'request_id': 'req-a9cb0e24-c894-4a60-9b9f-8e6751fe1ec8'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:53.090258 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a9cb0e24-c894-4a60-9b9f-8e6751fe1ec8 admin admin] The token cache is being invalidate because project d9f6e6209ad24963b27eb9a6aabbe092 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:21:53.090813 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a9cb0e24-c894-4a60-9b9f-8e6751fe1ec8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:53.091747 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a9cb0e24-c894-4a60-9b9f-8e6751fe1ec8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:53.096853 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 856/1727] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:21:52 2022] DELETE /identity/v3/projects/d9f6e6209ad24963b27eb9a6aabbe092 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:53.102075 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c8d14-4076-4541-be59-47b600221a11 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:53.102595 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c8d14-4076-4541-be59-47b600221a11 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:53.105716 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c8d14-4076-4541-be59-47b600221a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:53.148306 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-715c8d14-4076-4541-be59-47b600221a11 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:53.152502 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-715c8d14-4076-4541-be59-47b600221a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:53.204131 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:53.220293 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de6030-dbf4-4bb2-acb1-d53f649525df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:53.220954 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de6030-dbf4-4bb2-acb1-d53f649525df service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:53.221234 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de6030-dbf4-4bb2-acb1-d53f649525df service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:53.221711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de6030-dbf4-4bb2-acb1-d53f649525df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:53.232391 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-00de6030-dbf4-4bb2-acb1-d53f649525df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:53.382681 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 857/1728] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:53.387555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c8d14-4076-4541-be59-47b600221a11 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:53.392403 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-715c8d14-4076-4541-be59-47b600221a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:53.434537 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c8d14-4076-4541-be59-47b600221a11 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:53.474397 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b5573-7255-416e-a3fb-cd9bbfb9784b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:53.477530 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b5573-7255-416e-a3fb-cd9bbfb9784b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:53.479645 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b5573-7255-416e-a3fb-cd9bbfb9784b admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:53.479988 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b5573-7255-416e-a3fb-cd9bbfb9784b admin admin] PATH_INFO: `/v3/projects/3a0b9dd0d94644b1bbac5f379c8d021c` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:53.482861 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b5573-7255-416e-a3fb-cd9bbfb9784b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a0b9dd0d94644b1bbac5f379c8d021c)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:53.484896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b5573-7255-416e-a3fb-cd9bbfb9784b admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:53.495215 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de6030-dbf4-4bb2-acb1-d53f649525df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:53.497000 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de6030-dbf4-4bb2-acb1-d53f649525df service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:53.516859 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 872/1729] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:21:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 04:21:53.519321 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd9e5b-4120-40ae-afc9-ba13f973c940 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:53.519839 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd9e5b-4120-40ae-afc9-ba13f973c940 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:53.520207 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd9e5b-4120-40ae-afc9-ba13f973c940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:53.581508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-23bd9e5b-4120-40ae-afc9-ba13f973c940 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:53.587309 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-23bd9e5b-4120-40ae-afc9-ba13f973c940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:53.613824 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 873/1730] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:53.613824 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd9e5b-4120-40ae-afc9-ba13f973c940 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:53.615673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-23bd9e5b-4120-40ae-afc9-ba13f973c940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:53.723053 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-218b5573-7255-416e-a3fb-cd9bbfb9784b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a0b9dd0d94644b1bbac5f379c8d021c', 'request_id': 'req-218b5573-7255-416e-a3fb-cd9bbfb9784b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:53.751865 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-218b5573-7255-416e-a3fb-cd9bbfb9784b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a0b9dd0d94644b1bbac5f379c8d021c', 'request_id': 'req-218b5573-7255-416e-a3fb-cd9bbfb9784b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:53.766134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-218b5573-7255-416e-a3fb-cd9bbfb9784b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a0b9dd0d94644b1bbac5f379c8d021c', 'request_id': 'req-218b5573-7255-416e-a3fb-cd9bbfb9784b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:53.774584 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd9e5b-4120-40ae-afc9-ba13f973c940 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:53.787567 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-218b5573-7255-416e-a3fb-cd9bbfb9784b admin admin] The token cache is being invalidate because project 3a0b9dd0d94644b1bbac5f379c8d021c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:21:53.787865 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-218b5573-7255-416e-a3fb-cd9bbfb9784b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:53.788524 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-218b5573-7255-416e-a3fb-cd9bbfb9784b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:53.794115 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 858/1731] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:21:53 2022] DELETE /identity/v3/projects/3a0b9dd0d94644b1bbac5f379c8d021c => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:53.795029 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b5573-7255-416e-a3fb-cd9bbfb9784b admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:53.798851 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-218b5573-7255-416e-a3fb-cd9bbfb9784b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:53.893604 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e37ee3-b8b7-4853-a869-b9084d32370b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:53.894890 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e37ee3-b8b7-4853-a869-b9084d32370b service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:53.896157 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e37ee3-b8b7-4853-a869-b9084d32370b service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:53.897168 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e37ee3-b8b7-4853-a869-b9084d32370b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:53.901787 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a5e37ee3-b8b7-4853-a869-b9084d32370b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:53.911008 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:53.930328 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b5573-7255-416e-a3fb-cd9bbfb9784b admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:53.950512 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fce38cd-1fd4-4970-a8b0-b81412902433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:53.957676 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce38cd-1fd4-4970-a8b0-b81412902433 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:53.958038 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce38cd-1fd4-4970-a8b0-b81412902433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:53.958317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce38cd-1fd4-4970-a8b0-b81412902433 admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:53.961861 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce38cd-1fd4-4970-a8b0-b81412902433 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:21:53.962267 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce38cd-1fd4-4970-a8b0-b81412902433 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:53.968126 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce38cd-1fd4-4970-a8b0-b81412902433 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:53.999577 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 859/1732] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:21:54.003817 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595f61b-85fb-4437-9ef9-613292ba57f1 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:54.004206 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595f61b-85fb-4437-9ef9-613292ba57f1 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:54.004371 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595f61b-85fb-4437-9ef9-613292ba57f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:54.047677 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-2595f61b-85fb-4437-9ef9-613292ba57f1 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:54.050018 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2595f61b-85fb-4437-9ef9-613292ba57f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:54.068609 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:54.078671 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 860/1733] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:54.079594 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595f61b-85fb-4437-9ef9-613292ba57f1 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:54.084147 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e37ee3-b8b7-4853-a869-b9084d32370b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:54.085649 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e37ee3-b8b7-4853-a869-b9084d32370b service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:54.101912 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595f61b-85fb-4437-9ef9-613292ba57f1 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:54.108040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 874/1734] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:21:53 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:21:54.109964 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b176f-87bd-4a2c-9bf1-a2aeaa770d8e None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:54.110273 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b176f-87bd-4a2c-9bf1-a2aeaa770d8e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:54.110555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b176f-87bd-4a2c-9bf1-a2aeaa770d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:54.115993 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c866216-c60c-490f-8e27-693b47058ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:54.116583 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c866216-c60c-490f-8e27-693b47058ae2 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:54.116850 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c866216-c60c-490f-8e27-693b47058ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:54.117137 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c866216-c60c-490f-8e27-693b47058ae2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:54.117812 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c866216-c60c-490f-8e27-693b47058ae2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:54.120558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c866216-c60c-490f-8e27-693b47058ae2 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:54.150039 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-d26b176f-87bd-4a2c-9bf1-a2aeaa770d8e None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:54.152171 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d26b176f-87bd-4a2c-9bf1-a2aeaa770d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:54.162048 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 861/1735] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:21:54 2022] POST /identity/v3/projects => generated 373 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:54.162678 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 875/1736] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:54.165997 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c866216-c60c-490f-8e27-693b47058ae2 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:54.166468 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b176f-87bd-4a2c-9bf1-a2aeaa770d8e None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:54.168585 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-7c866216-c60c-490f-8e27-693b47058ae2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:54.176847 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b176f-87bd-4a2c-9bf1-a2aeaa770d8e None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:54.264150 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6022300-e976-4a86-b8f5-e971b9d434c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:54.264998 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6022300-e976-4a86-b8f5-e971b9d434c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:54.265337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6022300-e976-4a86-b8f5-e971b9d434c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:54.265705 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6022300-e976-4a86-b8f5-e971b9d434c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:54.266831 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6022300-e976-4a86-b8f5-e971b9d434c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:54.268483 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6022300-e976-4a86-b8f5-e971b9d434c4 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:54.273779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c866216-c60c-490f-8e27-693b47058ae2 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:54.292570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31fb430-c12e-4dc0-bb9f-9a7e735a299c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:54.293425 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31fb430-c12e-4dc0-bb9f-9a7e735a299c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:54.293719 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31fb430-c12e-4dc0-bb9f-9a7e735a299c admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:54.294159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31fb430-c12e-4dc0-bb9f-9a7e735a299c admin admin] PATH_INFO: `/v3/projects/1e375d85803c459c8905bdcd5a0c6b67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:54.295453 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31fb430-c12e-4dc0-bb9f-9a7e735a299c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e375d85803c459c8905bdcd5a0c6b67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:54.298401 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31fb430-c12e-4dc0-bb9f-9a7e735a299c admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:54.301080 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 876/1737] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:21:54 2022] POST /identity/v3/users => generated 287 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:54.302222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6022300-e976-4a86-b8f5-e971b9d434c4 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:54.315282 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6022300-e976-4a86-b8f5-e971b9d434c4 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:54.321144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ad8c3d-055c-4868-8938-c0fbda7a382e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:54.322075 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad8c3d-055c-4868-8938-c0fbda7a382e admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:54.322585 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad8c3d-055c-4868-8938-c0fbda7a382e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:54.323477 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad8c3d-055c-4868-8938-c0fbda7a382e admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:54.324351 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ad8c3d-055c-4868-8938-c0fbda7a382e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:21:54.324851 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ad8c3d-055c-4868-8938-c0fbda7a382e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:54.326838 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ad8c3d-055c-4868-8938-c0fbda7a382e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:54.345410 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 877/1738] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:21:54.351282 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ad8c3d-055c-4868-8938-c0fbda7a382e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:54.401914 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ad8c3d-055c-4868-8938-c0fbda7a382e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:54.458158 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-e31fb430-c12e-4dc0-bb9f-9a7e735a299c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e375d85803c459c8905bdcd5a0c6b67', 'request_id': 'req-e31fb430-c12e-4dc0-bb9f-9a7e735a299c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:54.486703 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-e31fb430-c12e-4dc0-bb9f-9a7e735a299c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e375d85803c459c8905bdcd5a0c6b67', 'request_id': 'req-e31fb430-c12e-4dc0-bb9f-9a7e735a299c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:54.514425 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-e31fb430-c12e-4dc0-bb9f-9a7e735a299c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e375d85803c459c8905bdcd5a0c6b67', 'request_id': 'req-e31fb430-c12e-4dc0-bb9f-9a7e735a299c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:54.562974 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-e31fb430-c12e-4dc0-bb9f-9a7e735a299c admin admin] The token cache is being invalidate because project 1e375d85803c459c8905bdcd5a0c6b67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:21:54.563322 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-e31fb430-c12e-4dc0-bb9f-9a7e735a299c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:54.568594 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-e31fb430-c12e-4dc0-bb9f-9a7e735a299c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:54.579149 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 862/1739] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:21:54 2022] DELETE /identity/v3/projects/1e375d85803c459c8905bdcd5a0c6b67 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:54.580388 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31fb430-c12e-4dc0-bb9f-9a7e735a299c admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:54.594325 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e31fb430-c12e-4dc0-bb9f-9a7e735a299c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:54.612808 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc9e9c9-cf12-4193-bf93-dd5efe7a37f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:54.613931 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9e9c9-cf12-4193-bf93-dd5efe7a37f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:54.615698 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9e9c9-cf12-4193-bf93-dd5efe7a37f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:54.615698 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9e9c9-cf12-4193-bf93-dd5efe7a37f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:54.615698 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc9e9c9-cf12-4193-bf93-dd5efe7a37f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:54.618397 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc9e9c9-cf12-4193-bf93-dd5efe7a37f2 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:54.649272 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 878/1740] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:54.649836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31fb430-c12e-4dc0-bb9f-9a7e735a299c admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:54.655287 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc9e9c9-cf12-4193-bf93-dd5efe7a37f2 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:54.663589 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-6fc9e9c9-cf12-4193-bf93-dd5efe7a37f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:54.667473 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b478b64f-642b-408a-a5d0-f2311d9daca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:54.668142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478b64f-642b-408a-a5d0-f2311d9daca7 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:54.668483 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478b64f-642b-408a-a5d0-f2311d9daca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:54.683073 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478b64f-642b-408a-a5d0-f2311d9daca7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:54.687763 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b478b64f-642b-408a-a5d0-f2311d9daca7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:54.696662 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b478b64f-642b-408a-a5d0-f2311d9daca7 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:54.698452 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc9e9c9-cf12-4193-bf93-dd5efe7a37f2 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:54.719427 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38056c0-4519-4249-b1ae-fb5550274c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:54.720265 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38056c0-4519-4249-b1ae-fb5550274c1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:54.720660 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38056c0-4519-4249-b1ae-fb5550274c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:54.721050 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38056c0-4519-4249-b1ae-fb5550274c1c admin admin] PATH_INFO: `/v3/users/ec19e8c7484241489edaa2c23f62a971` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:54.722591 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38056c0-4519-4249-b1ae-fb5550274c1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec19e8c7484241489edaa2c23f62a971)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:54.724903 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38056c0-4519-4249-b1ae-fb5550274c1c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:54.805455 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 863/1741] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:21:54 2022] POST /identity/v3/projects => generated 373 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:54.807880 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b478b64f-642b-408a-a5d0-f2311d9daca7 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:54.812856 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b478b64f-642b-408a-a5d0-f2311d9daca7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:54.871442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f38056c0-4519-4249-b1ae-fb5550274c1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec19e8c7484241489edaa2c23f62a971'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:54.872588 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f38056c0-4519-4249-b1ae-fb5550274c1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec19e8c7484241489edaa2c23f62a971'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:54.874731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f38056c0-4519-4249-b1ae-fb5550274c1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec19e8c7484241489edaa2c23f62a971'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:54.899876 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:54.910335 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f38056c0-4519-4249-b1ae-fb5550274c1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec19e8c7484241489edaa2c23f62a971'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:54.932391 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f38056c0-4519-4249-b1ae-fb5550274c1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec19e8c7484241489edaa2c23f62a971'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:54.960553 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 879/1742] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:21:54 2022] DELETE /identity/v3/users/ec19e8c7484241489edaa2c23f62a971 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:54.961693 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:54.968545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38056c0-4519-4249-b1ae-fb5550274c1c admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:54.970202 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f38056c0-4519-4249-b1ae-fb5550274c1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:55.131646 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b478b64f-642b-408a-a5d0-f2311d9daca7 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:55.142526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38056c0-4519-4249-b1ae-fb5550274c1c admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:55.144374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55640088-7710-4dab-bd35-f242d852b7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:55.145167 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55640088-7710-4dab-bd35-f242d852b7e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:55.145724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55640088-7710-4dab-bd35-f242d852b7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:55.146030 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55640088-7710-4dab-bd35-f242d852b7e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:55.148204 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55640088-7710-4dab-bd35-f242d852b7e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:55.151357 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55640088-7710-4dab-bd35-f242d852b7e5 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:55.152190 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f6d1e-4b53-4f5f-936c-69875608aca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:55.153315 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f6d1e-4b53-4f5f-936c-69875608aca3 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:55.153753 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f6d1e-4b53-4f5f-936c-69875608aca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:55.154176 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f6d1e-4b53-4f5f-936c-69875608aca3 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:55.155047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f6d1e-4b53-4f5f-936c-69875608aca3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:55.157838 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f6d1e-4b53-4f5f-936c-69875608aca3 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:55.185620 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 864/1743] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:55.186882 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55640088-7710-4dab-bd35-f242d852b7e5 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:55.189631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-55640088-7710-4dab-bd35-f242d852b7e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:55.202740 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 880/1744] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:21:54 2022] POST /identity/v3/users => generated 287 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:55.206469 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55640088-7710-4dab-bd35-f242d852b7e5 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:55.207455 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f6d1e-4b53-4f5f-936c-69875608aca3 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:55.211894 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5107550-4eb5-478e-a67f-73b452ffa8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:55.212588 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5107550-4eb5-478e-a67f-73b452ffa8cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:55.212921 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5107550-4eb5-478e-a67f-73b452ffa8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:55.213174 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5107550-4eb5-478e-a67f-73b452ffa8cf admin admin] PATH_INFO: `/v3/projects/dddb75e78b394e309f2039e042aaa4a8/users/6aa1364f2f0849acae25076210fb1973/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:55.221535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f6d1e-4b53-4f5f-936c-69875608aca3 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:55.227108 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6835990-ead1-4333-9a44-63fa4a1e9711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:55.228176 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6835990-ead1-4333-9a44-63fa4a1e9711 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:55.229128 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6835990-ead1-4333-9a44-63fa4a1e9711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:55.229514 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6835990-ead1-4333-9a44-63fa4a1e9711 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:55.231250 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6835990-ead1-4333-9a44-63fa4a1e9711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:55.232809 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6835990-ead1-4333-9a44-63fa4a1e9711 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:55.243684 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5107550-4eb5-478e-a67f-73b452ffa8cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=dddb75e78b394e309f2039e042aaa4a8, user_id=6aa1364f2f0849acae25076210fb1973, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:55.248691 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5107550-4eb5-478e-a67f-73b452ffa8cf admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:55.252442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 881/1745] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 04:21:55.256440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6835990-ead1-4333-9a44-63fa4a1e9711 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:55.270334 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6835990-ead1-4333-9a44-63fa4a1e9711 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:55.309314 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 865/1746] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:21:55 2022] PUT /identity/v3/projects/dddb75e78b394e309f2039e042aaa4a8/users/6aa1364f2f0849acae25076210fb1973/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:21:55.320534 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f922a-f9fd-49d4-8671-83fdcd6d8be4 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:55.323129 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f922a-f9fd-49d4-8671-83fdcd6d8be4 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:55.324166 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f922a-f9fd-49d4-8671-83fdcd6d8be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:55.378445 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-ca6f922a-f9fd-49d4-8671-83fdcd6d8be4 None None] MFA Rules not processed for user `6aa1364f2f0849acae25076210fb1973`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:55.381236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ca6f922a-f9fd-49d4-8671-83fdcd6d8be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:55.387384 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed4c0e-211b-4144-a66d-4903ea8f8325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:55.387614 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed4c0e-211b-4144-a66d-4903ea8f8325 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:55.387957 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed4c0e-211b-4144-a66d-4903ea8f8325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:55.388307 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed4c0e-211b-4144-a66d-4903ea8f8325 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:55.389092 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed4c0e-211b-4144-a66d-4903ea8f8325 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:55.391176 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed4c0e-211b-4144-a66d-4903ea8f8325 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:55.407901 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 882/1747] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:55.411380 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed4c0e-211b-4144-a66d-4903ea8f8325 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:55.446362 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed4c0e-211b-4144-a66d-4903ea8f8325 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:55.455033 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c255b42c-bdb8-4dbb-a4df-7df9d3216c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:55.455644 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c255b42c-bdb8-4dbb-a4df-7df9d3216c55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:55.455920 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c255b42c-bdb8-4dbb-a4df-7df9d3216c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:55.456188 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c255b42c-bdb8-4dbb-a4df-7df9d3216c55 admin admin] PATH_INFO: `/v3/projects/04b921e9d954424f9c1822e2fc77e36e/users/827504a764024c91ad266804b7b2e162/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:55.477952 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c255b42c-bdb8-4dbb-a4df-7df9d3216c55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04b921e9d954424f9c1822e2fc77e36e, user_id=827504a764024c91ad266804b7b2e162, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:55.481078 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c255b42c-bdb8-4dbb-a4df-7df9d3216c55 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:55.499835 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 883/1748] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:21:55 2022] PUT /identity/v3/projects/04b921e9d954424f9c1822e2fc77e36e/users/827504a764024c91ad266804b7b2e162/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:55.507430 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 866/1749] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:55.513095 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de47596-4c32-4ab4-b4ef-1f14a06499be None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:55.513716 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de47596-4c32-4ab4-b4ef-1f14a06499be None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:55.514764 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f922a-f9fd-49d4-8671-83fdcd6d8be4 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:55.515377 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de47596-4c32-4ab4-b4ef-1f14a06499be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:55.530901 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f922a-f9fd-49d4-8671-83fdcd6d8be4 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:55.552708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-6de47596-4c32-4ab4-b4ef-1f14a06499be None None] MFA Rules not processed for user `827504a764024c91ad266804b7b2e162`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:55.553924 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-6de47596-4c32-4ab4-b4ef-1f14a06499be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:55.568996 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:55.637724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200f381-922f-4f61-8304-4ea2f7bd3e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:55.638693 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200f381-922f-4f61-8304-4ea2f7bd3e80 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:55.639108 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200f381-922f-4f61-8304-4ea2f7bd3e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:55.639862 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200f381-922f-4f61-8304-4ea2f7bd3e80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:55.640772 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200f381-922f-4f61-8304-4ea2f7bd3e80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:55.643012 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200f381-922f-4f61-8304-4ea2f7bd3e80 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:55.653560 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 884/1750] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:55.654874 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de47596-4c32-4ab4-b4ef-1f14a06499be None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:55.656752 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-6de47596-4c32-4ab4-b4ef-1f14a06499be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:55.676051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de47596-4c32-4ab4-b4ef-1f14a06499be None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:55.680887 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b79e75-31d9-4353-b816-43043a4b38cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:55.681548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b79e75-31d9-4353-b816-43043a4b38cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:55.682036 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b79e75-31d9-4353-b816-43043a4b38cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:55.682340 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b79e75-31d9-4353-b816-43043a4b38cb admin admin] PATH_INFO: `/v3/projects/89c108cc7727445394cf435943a6a0b6` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:55.683547 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b79e75-31d9-4353-b816-43043a4b38cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=89c108cc7727445394cf435943a6a0b6)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:55.685504 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b79e75-31d9-4353-b816-43043a4b38cb admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:55.686860 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 867/1751] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:21:55 2022] POST /identity/v3/projects => generated 373 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:55.688624 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200f381-922f-4f61-8304-4ea2f7bd3e80 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:55.691261 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-4200f381-922f-4f61-8304-4ea2f7bd3e80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:55.719893 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:55.776924 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-61b79e75-31d9-4353-b816-43043a4b38cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89c108cc7727445394cf435943a6a0b6', 'request_id': 'req-61b79e75-31d9-4353-b816-43043a4b38cb'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:55.786490 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-61b79e75-31d9-4353-b816-43043a4b38cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89c108cc7727445394cf435943a6a0b6', 'request_id': 'req-61b79e75-31d9-4353-b816-43043a4b38cb'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:55.810001 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-61b79e75-31d9-4353-b816-43043a4b38cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89c108cc7727445394cf435943a6a0b6', 'request_id': 'req-61b79e75-31d9-4353-b816-43043a4b38cb'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:55.811549 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200f381-922f-4f61-8304-4ea2f7bd3e80 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:55.814381 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-61b79e75-31d9-4353-b816-43043a4b38cb admin admin] The token cache is being invalidate because project 89c108cc7727445394cf435943a6a0b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:21:55.814681 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-61b79e75-31d9-4353-b816-43043a4b38cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:55.815755 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-61b79e75-31d9-4353-b816-43043a4b38cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:55.822015 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 885/1752] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:21:55 2022] DELETE /identity/v3/projects/89c108cc7727445394cf435943a6a0b6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:55.822015 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b79e75-31d9-4353-b816-43043a4b38cb admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:55.823065 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-61b79e75-31d9-4353-b816-43043a4b38cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:55.895013 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a57face-272a-48a4-b2d7-16695992ce98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:55.895966 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57face-272a-48a4-b2d7-16695992ce98 service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:55.896351 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57face-272a-48a4-b2d7-16695992ce98 service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:55.896724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57face-272a-48a4-b2d7-16695992ce98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:55.898783 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9a57face-272a-48a4-b2d7-16695992ce98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:55.900083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b79e75-31d9-4353-b816-43043a4b38cb admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:55.905854 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963acce-7ced-4bd3-8227-81b0d1b2455e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:55.906810 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963acce-7ced-4bd3-8227-81b0d1b2455e admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:55.907105 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963acce-7ced-4bd3-8227-81b0d1b2455e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:55.907390 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963acce-7ced-4bd3-8227-81b0d1b2455e admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:55.908080 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963acce-7ced-4bd3-8227-81b0d1b2455e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:55.910040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963acce-7ced-4bd3-8227-81b0d1b2455e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:55.936924 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 886/1753] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:21:55 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:55.938247 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbb41a-4042-487d-80b5-0773f2eb5434 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:55.938542 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbb41a-4042-487d-80b5-0773f2eb5434 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:55.939207 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbb41a-4042-487d-80b5-0773f2eb5434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:55.961275 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-4bcbb41a-4042-487d-80b5-0773f2eb5434 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:55.962511 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-4bcbb41a-4042-487d-80b5-0773f2eb5434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:55.973306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 887/1754] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:55.974231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcbb41a-4042-487d-80b5-0773f2eb5434 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:55.985913 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcbb41a-4042-487d-80b5-0773f2eb5434 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:55.986356 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a57face-272a-48a4-b2d7-16695992ce98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:55.988342 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a57face-272a-48a4-b2d7-16695992ce98 service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:55.991231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a66ca-379b-473c-8ee3-fef19d498cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:55.991877 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a66ca-379b-473c-8ee3-fef19d498cd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:55.992162 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a66ca-379b-473c-8ee3-fef19d498cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:55.992443 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a66ca-379b-473c-8ee3-fef19d498cd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:55.993163 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a66ca-379b-473c-8ee3-fef19d498cd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:55.995100 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a66ca-379b-473c-8ee3-fef19d498cd3 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:56.000837 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 868/1755] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:21:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:21:56.002776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af14b4-2c2c-4a8b-acd7-d4cfbd234a2d None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:56.003070 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af14b4-2c2c-4a8b-acd7-d4cfbd234a2d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:56.003405 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af14b4-2c2c-4a8b-acd7-d4cfbd234a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:56.012345 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 888/1756] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:56.013115 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a66ca-379b-473c-8ee3-fef19d498cd3 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:56.026111 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a66ca-379b-473c-8ee3-fef19d498cd3 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:56.027799 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-f4af14b4-2c2c-4a8b-acd7-d4cfbd234a2d None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:56.029289 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f4af14b4-2c2c-4a8b-acd7-d4cfbd234a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:56.031528 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d5bd2-d4dc-40f3-b5c7-123667a6c774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:56.032492 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d5bd2-d4dc-40f3-b5c7-123667a6c774 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:56.033550 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d5bd2-d4dc-40f3-b5c7-123667a6c774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:56.033830 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d5bd2-d4dc-40f3-b5c7-123667a6c774 admin admin] PATH_INFO: `/v3/projects/91e698a6be8d4ea3a5e87b2b7f76883b/users/e193db74240b4ace8b3b3d17fbf75da4/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:56.039136 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 869/1757] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:56.043855 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af14b4-2c2c-4a8b-acd7-d4cfbd234a2d None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:56.044173 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:56.053925 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af14b4-2c2c-4a8b-acd7-d4cfbd234a2d None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:56.061241 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d5bd2-d4dc-40f3-b5c7-123667a6c774 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91e698a6be8d4ea3a5e87b2b7f76883b, user_id=e193db74240b4ace8b3b3d17fbf75da4, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:56.064455 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf75f2d-7bf4-447e-b3e0-ffa084ccc0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:56.064577 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d5bd2-d4dc-40f3-b5c7-123667a6c774 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:56.065099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf75f2d-7bf4-447e-b3e0-ffa084ccc0d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:56.065377 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf75f2d-7bf4-447e-b3e0-ffa084ccc0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:56.065655 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf75f2d-7bf4-447e-b3e0-ffa084ccc0d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:56.066408 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf75f2d-7bf4-447e-b3e0-ffa084ccc0d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:21:56.066719 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf75f2d-7bf4-447e-b3e0-ffa084ccc0d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:56.068330 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf75f2d-7bf4-447e-b3e0-ffa084ccc0d7 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:56.086154 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 889/1758] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:21:56 2022] PUT /identity/v3/projects/91e698a6be8d4ea3a5e87b2b7f76883b/users/e193db74240b4ace8b3b3d17fbf75da4/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:56.089886 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d5bd2-d4dc-40f3-b5c7-123667a6c774 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:56.093074 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 870/1759] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:21:56.094106 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf75f2d-7bf4-447e-b3e0-ffa084ccc0d7 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:56.120337 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf75f2d-7bf4-447e-b3e0-ffa084ccc0d7 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:56.194723 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:56.253959 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d5bd2-d4dc-40f3-b5c7-123667a6c774 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:56.262833 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c174de-caf9-4ce3-b0e3-3fc81d00b5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:56.264028 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c174de-caf9-4ce3-b0e3-3fc81d00b5ae admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:56.264421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c174de-caf9-4ce3-b0e3-3fc81d00b5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:56.264831 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c174de-caf9-4ce3-b0e3-3fc81d00b5ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:56.266122 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c174de-caf9-4ce3-b0e3-3fc81d00b5ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:56.268034 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c174de-caf9-4ce3-b0e3-3fc81d00b5ae admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:56.288993 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087fadab-91dd-4189-9b4d-93714c94b094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:56.289622 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087fadab-91dd-4189-9b4d-93714c94b094 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:56.290040 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087fadab-91dd-4189-9b4d-93714c94b094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:56.290540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087fadab-91dd-4189-9b4d-93714c94b094 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:56.291419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087fadab-91dd-4189-9b4d-93714c94b094 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:56.294808 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087fadab-91dd-4189-9b4d-93714c94b094 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:56.302524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 890/1760] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 04:21:56.307526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c174de-caf9-4ce3-b0e3-3fc81d00b5ae admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:56.331623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c174de-caf9-4ce3-b0e3-3fc81d00b5ae admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:56.340717 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7162423f-1920-476e-8941-c441790227aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:56.342518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162423f-1920-476e-8941-c441790227aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:56.342830 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162423f-1920-476e-8941-c441790227aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:56.343125 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162423f-1920-476e-8941-c441790227aa admin admin] PATH_INFO: `/v3/projects/91e698a6be8d4ea3a5e87b2b7f76883b/users/e193db74240b4ace8b3b3d17fbf75da4/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:56.348793 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7162423f-1920-476e-8941-c441790227aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=91e698a6be8d4ea3a5e87b2b7f76883b, user_id=e193db74240b4ace8b3b3d17fbf75da4, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:56.353878 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7162423f-1920-476e-8941-c441790227aa admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:56.358464 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 871/1761] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:21:56 2022] POST /identity/v3/projects => generated 381 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:56.359620 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087fadab-91dd-4189-9b4d-93714c94b094 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:56.375924 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 891/1762] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:21:56 2022] PUT /identity/v3/projects/91e698a6be8d4ea3a5e87b2b7f76883b/users/e193db74240b4ace8b3b3d17fbf75da4/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:56.387357 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282ceb8-6755-4ac9-9eed-53db65c4caf7 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:56.387665 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282ceb8-6755-4ac9-9eed-53db65c4caf7 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:56.388112 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282ceb8-6755-4ac9-9eed-53db65c4caf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:56.421531 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-5282ceb8-6755-4ac9-9eed-53db65c4caf7 None None] MFA Rules not processed for user `e193db74240b4ace8b3b3d17fbf75da4`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:56.424918 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5282ceb8-6755-4ac9-9eed-53db65c4caf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:56.476012 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087fadab-91dd-4189-9b4d-93714c94b094 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:56.538190 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 892/1763] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:56 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:21:56.547432 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282ceb8-6755-4ac9-9eed-53db65c4caf7 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:56.550510 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5282ceb8-6755-4ac9-9eed-53db65c4caf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:56.599808 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224e74cf-8e05-4f58-acef-439a11e90f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:56.600313 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e74cf-8e05-4f58-acef-439a11e90f58 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:56.600819 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e74cf-8e05-4f58-acef-439a11e90f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:56.601580 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e74cf-8e05-4f58-acef-439a11e90f58 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:56.602431 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224e74cf-8e05-4f58-acef-439a11e90f58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:56.618308 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224e74cf-8e05-4f58-acef-439a11e90f58 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:56.659316 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 872/1764] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:21:56 2022] POST /identity/v3/users => generated 291 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:21:56.661030 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224e74cf-8e05-4f58-acef-439a11e90f58 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:56.664519 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282ceb8-6755-4ac9-9eed-53db65c4caf7 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:56.668577 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49974cab-9acb-4396-8214-232172053a81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:56.669198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49974cab-9acb-4396-8214-232172053a81 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:56.669538 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49974cab-9acb-4396-8214-232172053a81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:56.670156 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49974cab-9acb-4396-8214-232172053a81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:56.673786 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224e74cf-8e05-4f58-acef-439a11e90f58 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:56.678220 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:56.679789 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc912b-ade0-47c0-aa52-4770e497e7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:56.680680 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc912b-ade0-47c0-aa52-4770e497e7e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:56.680808 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc912b-ade0-47c0-aa52-4770e497e7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:56.681050 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc912b-ade0-47c0-aa52-4770e497e7e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:56.682169 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bc912b-ade0-47c0-aa52-4770e497e7e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:56.683011 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49974cab-9acb-4396-8214-232172053a81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:56.684353 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bc912b-ade0-47c0-aa52-4770e497e7e2 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:56.686910 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49974cab-9acb-4396-8214-232172053a81 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:56.696776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 873/1765] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:56.702397 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc912b-ade0-47c0-aa52-4770e497e7e2 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:56.705405 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 893/1766] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:21:56 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:21:56.717124 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc912b-ade0-47c0-aa52-4770e497e7e2 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:56.723646 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66448f35-75bb-41b9-91a8-1bfbe95b38b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:56.724528 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66448f35-75bb-41b9-91a8-1bfbe95b38b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:56.724885 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66448f35-75bb-41b9-91a8-1bfbe95b38b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:56.725250 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66448f35-75bb-41b9-91a8-1bfbe95b38b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:56.726568 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66448f35-75bb-41b9-91a8-1bfbe95b38b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:56.728396 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66448f35-75bb-41b9-91a8-1bfbe95b38b7 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:56.747227 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 874/1767] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:56.756787 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49974cab-9acb-4396-8214-232172053a81 service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:56.769647 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49974cab-9acb-4396-8214-232172053a81 service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:56.774994 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc2c77-8d9d-45df-ba77-20852e178e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:56.776468 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc2c77-8d9d-45df-ba77-20852e178e65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:56.776959 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc2c77-8d9d-45df-ba77-20852e178e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:56.777140 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc2c77-8d9d-45df-ba77-20852e178e65 admin admin] PATH_INFO: `/v3/projects/097927e9acbb4dcba1e59f15045b6cc3/users/71bb6ca833704f3b9f8342fe99abb4b9/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:56.798286 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc2c77-8d9d-45df-ba77-20852e178e65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=097927e9acbb4dcba1e59f15045b6cc3, user_id=71bb6ca833704f3b9f8342fe99abb4b9, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:56.801080 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc2c77-8d9d-45df-ba77-20852e178e65 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:56.803359 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66448f35-75bb-41b9-91a8-1bfbe95b38b7 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:56.815476 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66448f35-75bb-41b9-91a8-1bfbe95b38b7 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:56.819107 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 894/1768] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:21:56 2022] PUT /identity/v3/projects/097927e9acbb4dcba1e59f15045b6cc3/users/71bb6ca833704f3b9f8342fe99abb4b9/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:56.830398 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1bc68-fda4-4150-9bf6-ee1499ba441f None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:56.830708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1bc68-fda4-4150-9bf6-ee1499ba441f None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:56.830993 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1bc68-fda4-4150-9bf6-ee1499ba441f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:56.847740 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:56.862092 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-fbf1bc68-fda4-4150-9bf6-ee1499ba441f None None] MFA Rules not processed for user `71bb6ca833704f3b9f8342fe99abb4b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:56.863307 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-fbf1bc68-fda4-4150-9bf6-ee1499ba441f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:56.876428 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ecc73-903c-441d-bd85-8093c23368ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:56.877058 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ecc73-903c-441d-bd85-8093c23368ff service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:56.877338 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ecc73-903c-441d-bd85-8093c23368ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:56.877611 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ecc73-903c-441d-bd85-8093c23368ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:56.879433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-df1ecc73-903c-441d-bd85-8093c23368ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:56.940948 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 895/1769] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:21:56.945605 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1bc68-fda4-4150-9bf6-ee1499ba441f None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:56.949445 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ecc73-903c-441d-bd85-8093c23368ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:56.950931 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ecc73-903c-441d-bd85-8093c23368ff service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:56.959830 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 875/1770] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:21:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:21:57.010426 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1bc68-fda4-4150-9bf6-ee1499ba441f None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:57.016584 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c67e21b-ef1c-4d8c-a059-3feb105b2079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:57.017257 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67e21b-ef1c-4d8c-a059-3feb105b2079 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:57.017887 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67e21b-ef1c-4d8c-a059-3feb105b2079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:57.017887 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67e21b-ef1c-4d8c-a059-3feb105b2079 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:57.018529 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c67e21b-ef1c-4d8c-a059-3feb105b2079 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:57.020810 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c67e21b-ef1c-4d8c-a059-3feb105b2079 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:57.055564 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 896/1771] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:21:56 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:21:57.059858 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ecc73-903c-441d-bd85-8093c23368ff service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:57.071433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ecc73-903c-441d-bd85-8093c23368ff service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:57.144542 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b11f76-a093-401d-833c-0a62bbc32bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:57.145195 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b11f76-a093-401d-833c-0a62bbc32bc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:57.146197 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b11f76-a093-401d-833c-0a62bbc32bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:57.146473 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b11f76-a093-401d-833c-0a62bbc32bc9 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:57.147150 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b11f76-a093-401d-833c-0a62bbc32bc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:57.149055 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b11f76-a093-401d-833c-0a62bbc32bc9 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:57.170719 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 876/1772] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:21:57 2022] POST /identity/v3/users => generated 299 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:21:57.174579 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c67e21b-ef1c-4d8c-a059-3feb105b2079 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:57.185284 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c67e21b-ef1c-4d8c-a059-3feb105b2079 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:57.190023 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d4674-5d8a-4346-b846-1f749afa3072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:57.190839 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d4674-5d8a-4346-b846-1f749afa3072 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:57.191149 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d4674-5d8a-4346-b846-1f749afa3072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:57.191544 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d4674-5d8a-4346-b846-1f749afa3072 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:57.192255 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d4674-5d8a-4346-b846-1f749afa3072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:57.193851 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d4674-5d8a-4346-b846-1f749afa3072 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:57.205142 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 897/1773] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:21:57.209211 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b11f76-a093-401d-833c-0a62bbc32bc9 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:57.219294 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b11f76-a093-401d-833c-0a62bbc32bc9 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:57.223948 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1f97e-205c-4b31-b97a-0fb917c4ff3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:57.224613 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1f97e-205c-4b31-b97a-0fb917c4ff3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:57.224898 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1f97e-205c-4b31-b97a-0fb917c4ff3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:57.225173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1f97e-205c-4b31-b97a-0fb917c4ff3f admin admin] PATH_INFO: `/v3/projects/5b25797600d14e538e35108ef00ed07a/users/8d8a390a6fc641d7b73b5f575a3bb14c/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:57.239903 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1f97e-205c-4b31-b97a-0fb917c4ff3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b25797600d14e538e35108ef00ed07a, user_id=8d8a390a6fc641d7b73b5f575a3bb14c, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:57.242979 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1f97e-205c-4b31-b97a-0fb917c4ff3f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:57.257134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 877/1774] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:21:57 2022] PUT /identity/v3/projects/5b25797600d14e538e35108ef00ed07a/users/8d8a390a6fc641d7b73b5f575a3bb14c/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:57.261966 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d4674-5d8a-4346-b846-1f749afa3072 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:57.272052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d4674-5d8a-4346-b846-1f749afa3072 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:57.305119 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:57.332490 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e59c5-d306-4ba1-a576-6e32a615895a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:57.333143 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e59c5-d306-4ba1-a576-6e32a615895a admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:57.333417 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e59c5-d306-4ba1-a576-6e32a615895a admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:57.333697 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e59c5-d306-4ba1-a576-6e32a615895a admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:57.334435 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4e59c5-d306-4ba1-a576-6e32a615895a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:57.336111 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4e59c5-d306-4ba1-a576-6e32a615895a admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:57.346552 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 898/1775] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 04:21:57.350797 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1f97e-205c-4b31-b97a-0fb917c4ff3f admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:57.361495 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1f97e-205c-4b31-b97a-0fb917c4ff3f admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:57.366132 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f2e7b-6c16-4151-aa26-c879fea30da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:57.366799 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f2e7b-6c16-4151-aa26-c879fea30da2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:57.367081 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f2e7b-6c16-4151-aa26-c879fea30da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:57.367357 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f2e7b-6c16-4151-aa26-c879fea30da2 admin admin] PATH_INFO: `/v3/projects/5b25797600d14e538e35108ef00ed07a/users/8d8a390a6fc641d7b73b5f575a3bb14c/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:57.369870 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f2e7b-6c16-4151-aa26-c879fea30da2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b25797600d14e538e35108ef00ed07a, user_id=8d8a390a6fc641d7b73b5f575a3bb14c, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:57.372856 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f2e7b-6c16-4151-aa26-c879fea30da2 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:57.385743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 878/1776] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:21:57 2022] PUT /identity/v3/projects/5b25797600d14e538e35108ef00ed07a/users/8d8a390a6fc641d7b73b5f575a3bb14c/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:21:57.397002 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea244c-21c9-4d98-a07d-a9139e70ee18 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:57.397301 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea244c-21c9-4d98-a07d-a9139e70ee18 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:57.397583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea244c-21c9-4d98-a07d-a9139e70ee18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:57.426248 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-45ea244c-21c9-4d98-a07d-a9139e70ee18 None None] MFA Rules not processed for user `8d8a390a6fc641d7b73b5f575a3bb14c`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:57.427458 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-45ea244c-21c9-4d98-a07d-a9139e70ee18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:57.499362 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 899/1777] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:57 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:57.510702 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f2e7b-6c16-4151-aa26-c879fea30da2 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:57.512503 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b30f2e7b-6c16-4151-aa26-c879fea30da2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:57.554190 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:57.572934 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f2e7b-6c16-4151-aa26-c879fea30da2 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:57.576499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bedd215-1183-495c-bdb8-a16bdf9a5815 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:57.577119 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bedd215-1183-495c-bdb8-a16bdf9a5815 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:57.577397 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bedd215-1183-495c-bdb8-a16bdf9a5815 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:57.577667 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bedd215-1183-495c-bdb8-a16bdf9a5815 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:57.586388 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bedd215-1183-495c-bdb8-a16bdf9a5815 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:57.587860 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bedd215-1183-495c-bdb8-a16bdf9a5815 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:57.597527 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 879/1778] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:21:57 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:21:57.663723 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea244c-21c9-4d98-a07d-a9139e70ee18 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:57.731705 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea244c-21c9-4d98-a07d-a9139e70ee18 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:57.737271 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6efc58f-1bae-48cb-9626-c8334e291aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:57.737928 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6efc58f-1bae-48cb-9626-c8334e291aa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:57.738302 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6efc58f-1bae-48cb-9626-c8334e291aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:57.738628 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6efc58f-1bae-48cb-9626-c8334e291aa6 admin admin] PATH_INFO: `/v3/users/71bb6ca833704f3b9f8342fe99abb4b9` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:57.739827 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6efc58f-1bae-48cb-9626-c8334e291aa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71bb6ca833704f3b9f8342fe99abb4b9)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:57.742019 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6efc58f-1bae-48cb-9626-c8334e291aa6 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:57.807011 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c6efc58f-1bae-48cb-9626-c8334e291aa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71bb6ca833704f3b9f8342fe99abb4b9'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:57.808215 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c6efc58f-1bae-48cb-9626-c8334e291aa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71bb6ca833704f3b9f8342fe99abb4b9'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:57.809234 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c6efc58f-1bae-48cb-9626-c8334e291aa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71bb6ca833704f3b9f8342fe99abb4b9'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:57.826438 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:57.834384 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c6efc58f-1bae-48cb-9626-c8334e291aa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71bb6ca833704f3b9f8342fe99abb4b9'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:57.845017 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c6efc58f-1bae-48cb-9626-c8334e291aa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71bb6ca833704f3b9f8342fe99abb4b9'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:57.864806 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 900/1779] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:21:57 2022] DELETE /identity/v3/users/71bb6ca833704f3b9f8342fe99abb4b9 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:57.870525 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59493caf-f3d6-40c2-8589-f0d97f47b965 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:57.871040 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59493caf-f3d6-40c2-8589-f0d97f47b965 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:57.871121 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59493caf-f3d6-40c2-8589-f0d97f47b965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:57.894622 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-59493caf-f3d6-40c2-8589-f0d97f47b965 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:57.895923 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-59493caf-f3d6-40c2-8589-f0d97f47b965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:57.996246 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 880/1780] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:58.011394 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6efc58f-1bae-48cb-9626-c8334e291aa6 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:58.014053 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c6efc58f-1bae-48cb-9626-c8334e291aa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:58.090547 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6efc58f-1bae-48cb-9626-c8334e291aa6 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:58.094786 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569334e-7850-4d78-9f4e-2efd7f4e5e80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:58.095605 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569334e-7850-4d78-9f4e-2efd7f4e5e80 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:58.096006 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569334e-7850-4d78-9f4e-2efd7f4e5e80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:58.097075 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569334e-7850-4d78-9f4e-2efd7f4e5e80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:58.157102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0569334e-7850-4d78-9f4e-2efd7f4e5e80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:58.159509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0569334e-7850-4d78-9f4e-2efd7f4e5e80 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:58.186364 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62874b3-fe70-4e00-978a-e182408ba97e None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:58.186876 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62874b3-fe70-4e00-978a-e182408ba97e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:58.187521 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62874b3-fe70-4e00-978a-e182408ba97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:58.196340 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 901/1781] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:21:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:21:58.371250 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-f62874b3-fe70-4e00-978a-e182408ba97e None None] MFA Rules not processed for user `8ec07ee1efba4e8089b94f24be338f86`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:58.372582 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f62874b3-fe70-4e00-978a-e182408ba97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:58.538639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 881/1782] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Jul 27 04:21:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:58.785091 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569334e-7850-4d78-9f4e-2efd7f4e5e80 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:58.787664 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-0569334e-7850-4d78-9f4e-2efd7f4e5e80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:58.856252 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62874b3-fe70-4e00-978a-e182408ba97e None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:58.858742 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f62874b3-fe70-4e00-978a-e182408ba97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:58.887158 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62874b3-fe70-4e00-978a-e182408ba97e None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:58.901732 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c65382-a59c-458b-ab8c-8173e61fd23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:58.902837 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c65382-a59c-458b-ab8c-8173e61fd23d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:58.903314 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c65382-a59c-458b-ab8c-8173e61fd23d admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:58.904089 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c65382-a59c-458b-ab8c-8173e61fd23d admin admin] PATH_INFO: `/v3/users/b582f8afe083473f9f9f1a722d791e89` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:58.905685 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c65382-a59c-458b-ab8c-8173e61fd23d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b582f8afe083473f9f9f1a722d791e89)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:58.908400 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c65382-a59c-458b-ab8c-8173e61fd23d admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:59.036962 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:59.040161 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-97c65382-a59c-458b-ab8c-8173e61fd23d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b582f8afe083473f9f9f1a722d791e89'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:59.040875 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-97c65382-a59c-458b-ab8c-8173e61fd23d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b582f8afe083473f9f9f1a722d791e89'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:59.060697 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-97c65382-a59c-458b-ab8c-8173e61fd23d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b582f8afe083473f9f9f1a722d791e89'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:59.095168 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-97c65382-a59c-458b-ab8c-8173e61fd23d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b582f8afe083473f9f9f1a722d791e89'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:59.110010 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569334e-7850-4d78-9f4e-2efd7f4e5e80 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:59.126138 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-97c65382-a59c-458b-ab8c-8173e61fd23d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b582f8afe083473f9f9f1a722d791e89'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:59.126754 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e76628-c68a-4e65-bd0f-0663529ad232 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57b5fa7517034a39930ce7c9d12e1875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:59.127305 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e76628-c68a-4e65-bd0f-0663529ad232 service swift] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:59.127592 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e76628-c68a-4e65-bd0f-0663529ad232 service swift] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:59.127869 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e76628-c68a-4e65-bd0f-0663529ad232 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:59.140508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-70e76628-c68a-4e65-bd0f-0663529ad232 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:59.151893 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 882/1783] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:21:58 2022] DELETE /identity/v3/users/b582f8afe083473f9f9f1a722d791e89 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:59.160211 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1619b4-de36-4a0a-806c-a3e8084895bf None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:59.161772 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1619b4-de36-4a0a-806c-a3e8084895bf None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:59.162179 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1619b4-de36-4a0a-806c-a3e8084895bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:59.167167 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e76628-c68a-4e65-bd0f-0663529ad232 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:59.168944 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e76628-c68a-4e65-bd0f-0663529ad232 service swift] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:59.213697 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 902/1784] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 27 04:21:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 04:21:59.235640 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-9e1619b4-de36-4a0a-806c-a3e8084895bf None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:59.235640 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9e1619b4-de36-4a0a-806c-a3e8084895bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:59.474030 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 883/1785] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:21:59.500549 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e76628-c68a-4e65-bd0f-0663529ad232 service swift] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:59.502243 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-70e76628-c68a-4e65-bd0f-0663529ad232 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:59.509063 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1619b4-de36-4a0a-806c-a3e8084895bf None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:21:59.512956 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9e1619b4-de36-4a0a-806c-a3e8084895bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:59.585076 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e76628-c68a-4e65-bd0f-0663529ad232 service swift] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:59.610914 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42715628-9aaa-407e-868e-77c8305616fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:59.612963 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42715628-9aaa-407e-868e-77c8305616fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:59.613263 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42715628-9aaa-407e-868e-77c8305616fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:59.613545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42715628-9aaa-407e-868e-77c8305616fb admin admin] PATH_INFO: `/v3/users/aa6ec6714d3f4836a603fad8174b1b62` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:59.627331 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42715628-9aaa-407e-868e-77c8305616fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa6ec6714d3f4836a603fad8174b1b62)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:59.629366 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42715628-9aaa-407e-868e-77c8305616fb admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:59.788932 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:21:59.795656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-42715628-9aaa-407e-868e-77c8305616fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa6ec6714d3f4836a603fad8174b1b62'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:59.796511 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-42715628-9aaa-407e-868e-77c8305616fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa6ec6714d3f4836a603fad8174b1b62'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:59.798335 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-42715628-9aaa-407e-868e-77c8305616fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa6ec6714d3f4836a603fad8174b1b62'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:59.839826 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1619b4-de36-4a0a-806c-a3e8084895bf None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:21:59.851660 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77611fe4-9c2b-4750-a31a-cfd2c1bbd7d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:21:59.852316 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77611fe4-9c2b-4750-a31a-cfd2c1bbd7d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:59.852967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77611fe4-9c2b-4750-a31a-cfd2c1bbd7d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:59.853250 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77611fe4-9c2b-4750-a31a-cfd2c1bbd7d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:59.855248 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-42715628-9aaa-407e-868e-77c8305616fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa6ec6714d3f4836a603fad8174b1b62'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:59.859623 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-77611fe4-9c2b-4750-a31a-cfd2c1bbd7d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:59.871643 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-42715628-9aaa-407e-868e-77c8305616fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa6ec6714d3f4836a603fad8174b1b62'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:21:59.892218 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77611fe4-9c2b-4750-a31a-cfd2c1bbd7d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:21:59.896186 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77611fe4-9c2b-4750-a31a-cfd2c1bbd7d3 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:21:59.901895 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 903/1786] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:21:59 2022] DELETE /identity/v3/users/aa6ec6714d3f4836a603fad8174b1b62 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:21:59.906061 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7c099-7aa5-40a6-93bf-f1a0b42cfb4e None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:59.906333 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7c099-7aa5-40a6-93bf-f1a0b42cfb4e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:59.906612 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7c099-7aa5-40a6-93bf-f1a0b42cfb4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:59.919419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 884/1787] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:21:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 04:21:59.923368 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb41d1-874b-4362-ae05-a5fadc478875 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:21:59.923611 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb41d1-874b-4362-ae05-a5fadc478875 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:21:59.923886 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb41d1-874b-4362-ae05-a5fadc478875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:21:59.945729 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-f5c7c099-7aa5-40a6-93bf-f1a0b42cfb4e None None] MFA Rules not processed for user `b4847f3736d844d78a206b7eab5f5fcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:59.947712 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f5c7c099-7aa5-40a6-93bf-f1a0b42cfb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:21:59.989599 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-18fb41d1-874b-4362-ae05-a5fadc478875 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:21:59.991701 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-18fb41d1-874b-4362-ae05-a5fadc478875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:00.144516 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 904/1788] 10.222.0.27 () {58 vars in 1015 bytes} [Wed Jul 27 04:21:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:00.203462 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 885/1789] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:21:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:00.469369 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7c099-7aa5-40a6-93bf-f1a0b42cfb4e None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:00.471857 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f5c7c099-7aa5-40a6-93bf-f1a0b42cfb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:00.516945 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7c099-7aa5-40a6-93bf-f1a0b42cfb4e None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:00.522639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb41d1-874b-4362-ae05-a5fadc478875 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:00.524515 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-18fb41d1-874b-4362-ae05-a5fadc478875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:00.539503 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525d523-b616-4762-90ce-4baa613d9d57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:00.541913 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525d523-b616-4762-90ce-4baa613d9d57 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:00.545186 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525d523-b616-4762-90ce-4baa613d9d57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:00.547418 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525d523-b616-4762-90ce-4baa613d9d57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:00.554670 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb41d1-874b-4362-ae05-a5fadc478875 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:00.576715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2f8c6-bf95-4f7a-aa2c-a36a42cd4bbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:00.577723 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525d523-b616-4762-90ce-4baa613d9d57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:00.580212 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525d523-b616-4762-90ce-4baa613d9d57 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:00.586655 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2f8c6-bf95-4f7a-aa2c-a36a42cd4bbd service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:00.586949 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2f8c6-bf95-4f7a-aa2c-a36a42cd4bbd service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:00.587231 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2f8c6-bf95-4f7a-aa2c-a36a42cd4bbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:00.598332 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:00.606834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc2f8c6-bf95-4f7a-aa2c-a36a42cd4bbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:00.609267 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc2f8c6-bf95-4f7a-aa2c-a36a42cd4bbd service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:00.612823 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 905/1790] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:00.639534 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 886/1791] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:00.654046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525d523-b616-4762-90ce-4baa613d9d57 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:00.655492 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a525d523-b616-4762-90ce-4baa613d9d57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:00.744870 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2f8c6-bf95-4f7a-aa2c-a36a42cd4bbd service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:00.746642 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-5cc2f8c6-bf95-4f7a-aa2c-a36a42cd4bbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:00.778845 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2f8c6-bf95-4f7a-aa2c-a36a42cd4bbd service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:00.794678 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc6d517-07f6-4b1c-9418-e990e35c4128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:00.795406 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6d517-07f6-4b1c-9418-e990e35c4128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:00.795797 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6d517-07f6-4b1c-9418-e990e35c4128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:00.796834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6d517-07f6-4b1c-9418-e990e35c4128 admin admin] PATH_INFO: `/v3/projects/097927e9acbb4dcba1e59f15045b6cc3` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:00.803417 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc6d517-07f6-4b1c-9418-e990e35c4128 admin admin] RBAC: Authorizing `identity:delete_project(project_id=097927e9acbb4dcba1e59f15045b6cc3)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:00.811047 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc6d517-07f6-4b1c-9418-e990e35c4128 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:00.815382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525d523-b616-4762-90ce-4baa613d9d57 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:00.894370 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf876b-f41a-473a-a234-2d97e5d0b266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:00.899168 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf876b-f41a-473a-a234-2d97e5d0b266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:00.904145 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf876b-f41a-473a-a234-2d97e5d0b266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:00.906944 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf876b-f41a-473a-a234-2d97e5d0b266 admin admin] PATH_INFO: `/v3/users/827504a764024c91ad266804b7b2e162` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:00.909161 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf876b-f41a-473a-a234-2d97e5d0b266 admin admin] RBAC: Authorizing `identity:delete_user(user_id=827504a764024c91ad266804b7b2e162)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:00.920212 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf876b-f41a-473a-a234-2d97e5d0b266 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:00.957479 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-efc6d517-07f6-4b1c-9418-e990e35c4128 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '097927e9acbb4dcba1e59f15045b6cc3', 'request_id': 'req-efc6d517-07f6-4b1c-9418-e990e35c4128'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:00.971432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-efc6d517-07f6-4b1c-9418-e990e35c4128 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '097927e9acbb4dcba1e59f15045b6cc3', 'request_id': 'req-efc6d517-07f6-4b1c-9418-e990e35c4128'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:00.981345 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-efc6d517-07f6-4b1c-9418-e990e35c4128 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '097927e9acbb4dcba1e59f15045b6cc3', 'request_id': 'req-efc6d517-07f6-4b1c-9418-e990e35c4128'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:01.012313 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-efc6d517-07f6-4b1c-9418-e990e35c4128 admin admin] The token cache is being invalidate because project 097927e9acbb4dcba1e59f15045b6cc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:01.013247 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-efc6d517-07f6-4b1c-9418-e990e35c4128 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:01.015289 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-efc6d517-07f6-4b1c-9418-e990e35c4128 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:01.020915 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 887/1792] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:00 2022] DELETE /identity/v3/projects/097927e9acbb4dcba1e59f15045b6cc3 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:22:01.022632 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc6d517-07f6-4b1c-9418-e990e35c4128 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:01.024880 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-efc6d517-07f6-4b1c-9418-e990e35c4128 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:01.066299 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9fdf876b-f41a-473a-a234-2d97e5d0b266 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '827504a764024c91ad266804b7b2e162'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:01.067004 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9fdf876b-f41a-473a-a234-2d97e5d0b266 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '827504a764024c91ad266804b7b2e162'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:01.069194 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9fdf876b-f41a-473a-a234-2d97e5d0b266 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '827504a764024c91ad266804b7b2e162'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:01.106503 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9fdf876b-f41a-473a-a234-2d97e5d0b266 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '827504a764024c91ad266804b7b2e162'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:01.118978 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9fdf876b-f41a-473a-a234-2d97e5d0b266 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '827504a764024c91ad266804b7b2e162'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:01.150979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 906/1793] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:00 2022] DELETE /identity/v3/users/827504a764024c91ad266804b7b2e162 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:01.155556 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b632cab7-a52f-4174-aa30-2bf860714adc None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:01.155882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b632cab7-a52f-4174-aa30-2bf860714adc None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:01.156136 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b632cab7-a52f-4174-aa30-2bf860714adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:01.193936 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-b632cab7-a52f-4174-aa30-2bf860714adc None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:01.196101 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b632cab7-a52f-4174-aa30-2bf860714adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:01.203842 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc6d517-07f6-4b1c-9418-e990e35c4128 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:01.282396 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:01.324596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd1b22-6c91-4436-9c4e-a8b60ab9d69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:01.325269 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd1b22-6c91-4436-9c4e-a8b60ab9d69b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:01.325552 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd1b22-6c91-4436-9c4e-a8b60ab9d69b admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:01.325828 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd1b22-6c91-4436-9c4e-a8b60ab9d69b admin admin] PATH_INFO: `/v3/users/8d8a390a6fc641d7b73b5f575a3bb14c` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:01.329177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd1b22-6c91-4436-9c4e-a8b60ab9d69b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d8a390a6fc641d7b73b5f575a3bb14c)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:01.331932 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd1b22-6c91-4436-9c4e-a8b60ab9d69b admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:01.362075 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 907/1794] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:01.363031 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b632cab7-a52f-4174-aa30-2bf860714adc None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:01.365692 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b632cab7-a52f-4174-aa30-2bf860714adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:01.386654 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:01.398178 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b632cab7-a52f-4174-aa30-2bf860714adc None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:01.409200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164d2e9-4bee-4bb8-b301-c84d8f00c3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:01.410035 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164d2e9-4bee-4bb8-b301-c84d8f00c3e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:01.410328 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164d2e9-4bee-4bb8-b301-c84d8f00c3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:01.410670 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164d2e9-4bee-4bb8-b301-c84d8f00c3e2 admin admin] PATH_INFO: `/v3/projects/8ed40de01a1f47b39b6c122bec5013c8` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:01.412365 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9164d2e9-4bee-4bb8-b301-c84d8f00c3e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ed40de01a1f47b39b6c122bec5013c8)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:01.414648 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f2dd1b22-6c91-4436-9c4e-a8b60ab9d69b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d8a390a6fc641d7b73b5f575a3bb14c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:01.415566 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f2dd1b22-6c91-4436-9c4e-a8b60ab9d69b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d8a390a6fc641d7b73b5f575a3bb14c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:01.416940 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9164d2e9-4bee-4bb8-b301-c84d8f00c3e2 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:01.430527 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f2dd1b22-6c91-4436-9c4e-a8b60ab9d69b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d8a390a6fc641d7b73b5f575a3bb14c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:01.449530 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f2dd1b22-6c91-4436-9c4e-a8b60ab9d69b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d8a390a6fc641d7b73b5f575a3bb14c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:01.464735 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f2dd1b22-6c91-4436-9c4e-a8b60ab9d69b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d8a390a6fc641d7b73b5f575a3bb14c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:01.477244 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 888/1795] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:01 2022] DELETE /identity/v3/users/8d8a390a6fc641d7b73b5f575a3bb14c => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:01.477558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd1b22-6c91-4436-9c4e-a8b60ab9d69b admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:01.479704 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f2dd1b22-6c91-4436-9c4e-a8b60ab9d69b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:01.491869 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9164d2e9-4bee-4bb8-b301-c84d8f00c3e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ed40de01a1f47b39b6c122bec5013c8', 'request_id': 'req-9164d2e9-4bee-4bb8-b301-c84d8f00c3e2'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:01.502693 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9164d2e9-4bee-4bb8-b301-c84d8f00c3e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ed40de01a1f47b39b6c122bec5013c8', 'request_id': 'req-9164d2e9-4bee-4bb8-b301-c84d8f00c3e2'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:01.513400 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9164d2e9-4bee-4bb8-b301-c84d8f00c3e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ed40de01a1f47b39b6c122bec5013c8', 'request_id': 'req-9164d2e9-4bee-4bb8-b301-c84d8f00c3e2'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:01.537874 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9164d2e9-4bee-4bb8-b301-c84d8f00c3e2 admin admin] The token cache is being invalidate because project 8ed40de01a1f47b39b6c122bec5013c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:01.538218 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9164d2e9-4bee-4bb8-b301-c84d8f00c3e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:01.539292 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9164d2e9-4bee-4bb8-b301-c84d8f00c3e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:01.546459 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 908/1796] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:01 2022] DELETE /identity/v3/projects/8ed40de01a1f47b39b6c122bec5013c8 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:01.548008 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2a892-b63c-469d-8a43-35d452c65ced None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:01.548295 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2a892-b63c-469d-8a43-35d452c65ced None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:01.548573 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2a892-b63c-469d-8a43-35d452c65ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:01.563793 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd1b22-6c91-4436-9c4e-a8b60ab9d69b admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:01.568800 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c93b3d-e623-44b7-a59d-f02e618e80bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:01.569810 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c93b3d-e623-44b7-a59d-f02e618e80bf service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:01.570292 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c93b3d-e623-44b7-a59d-f02e618e80bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:01.570737 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c93b3d-e623-44b7-a59d-f02e618e80bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:01.573630 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-60c93b3d-e623-44b7-a59d-f02e618e80bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:01.577365 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-47a2a892-b63c-469d-8a43-35d452c65ced None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:01.578864 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-47a2a892-b63c-469d-8a43-35d452c65ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:01.654734 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c93b3d-e623-44b7-a59d-f02e618e80bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:01.657358 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c93b3d-e623-44b7-a59d-f02e618e80bf service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:01.670609 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 889/1797] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:01.671975 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 909/1798] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:22:01.673407 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118375e6-d64d-4125-a81f-9a8908c7c10b None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:01.674081 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118375e6-d64d-4125-a81f-9a8908c7c10b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:01.674623 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118375e6-d64d-4125-a81f-9a8908c7c10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:01.700676 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-118375e6-d64d-4125-a81f-9a8908c7c10b None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:01.702203 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-118375e6-d64d-4125-a81f-9a8908c7c10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:01.714699 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 890/1799] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:22:01.719570 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a2a892-b63c-469d-8a43-35d452c65ced None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:01.730968 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a2a892-b63c-469d-8a43-35d452c65ced None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:01.738380 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8dd910-9773-4d73-be24-30ecdef263ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:01.739546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8dd910-9773-4d73-be24-30ecdef263ab admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:01.740055 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8dd910-9773-4d73-be24-30ecdef263ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:01.740541 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8dd910-9773-4d73-be24-30ecdef263ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:01.741652 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8dd910-9773-4d73-be24-30ecdef263ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:22:01.742183 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8dd910-9773-4d73-be24-30ecdef263ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:01.746420 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8dd910-9773-4d73-be24-30ecdef263ab admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:01.780765 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 910/1800] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:22:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:22:01.794011 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118375e6-d64d-4125-a81f-9a8908c7c10b None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:01.798329 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118375e6-d64d-4125-a81f-9a8908c7c10b None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:01.807834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca62028-4bae-427a-8a88-d523d41d6f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:01.809244 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca62028-4bae-427a-8a88-d523d41d6f80 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:01.809796 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca62028-4bae-427a-8a88-d523d41d6f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:01.810199 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca62028-4bae-427a-8a88-d523d41d6f80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:01.811480 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca62028-4bae-427a-8a88-d523d41d6f80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:01.814753 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca62028-4bae-427a-8a88-d523d41d6f80 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:01.816005 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8dd910-9773-4d73-be24-30ecdef263ab admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:01.818172 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5f8dd910-9773-4d73-be24-30ecdef263ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:01.833433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8dd910-9773-4d73-be24-30ecdef263ab admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:01.841915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d4a0bd-9016-426c-8297-3d68fe36a82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:01.842710 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4a0bd-9016-426c-8297-3d68fe36a82e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:01.843307 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4a0bd-9016-426c-8297-3d68fe36a82e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:01.843515 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4a0bd-9016-426c-8297-3d68fe36a82e admin admin] PATH_INFO: `/v3/projects/1bd488a9ff8f4e53a3d839ada563f5c1` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:01.848240 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d4a0bd-9016-426c-8297-3d68fe36a82e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bd488a9ff8f4e53a3d839ada563f5c1)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:01.863986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d4a0bd-9016-426c-8297-3d68fe36a82e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:01.898214 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 891/1801] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:01 2022] POST /identity/v3/projects => generated 357 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:01.903722 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca62028-4bae-427a-8a88-d523d41d6f80 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:01.964639 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:01.988021 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:01.992146 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-89d4a0bd-9016-426c-8297-3d68fe36a82e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bd488a9ff8f4e53a3d839ada563f5c1', 'request_id': 'req-89d4a0bd-9016-426c-8297-3d68fe36a82e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:02.003188 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-89d4a0bd-9016-426c-8297-3d68fe36a82e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bd488a9ff8f4e53a3d839ada563f5c1', 'request_id': 'req-89d4a0bd-9016-426c-8297-3d68fe36a82e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:02.020798 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-89d4a0bd-9016-426c-8297-3d68fe36a82e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bd488a9ff8f4e53a3d839ada563f5c1', 'request_id': 'req-89d4a0bd-9016-426c-8297-3d68fe36a82e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:02.044203 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-89d4a0bd-9016-426c-8297-3d68fe36a82e admin admin] The token cache is being invalidate because project 1bd488a9ff8f4e53a3d839ada563f5c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:02.044703 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-89d4a0bd-9016-426c-8297-3d68fe36a82e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:02.045700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-89d4a0bd-9016-426c-8297-3d68fe36a82e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:02.051110 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 911/1802] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:01 2022] DELETE /identity/v3/projects/1bd488a9ff8f4e53a3d839ada563f5c1 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:02.052540 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d4a0bd-9016-426c-8297-3d68fe36a82e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:02.060004 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-89d4a0bd-9016-426c-8297-3d68fe36a82e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:02.063657 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca62028-4bae-427a-8a88-d523d41d6f80 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:02.173670 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a00aae-5f18-4005-8dc8-bd4b2e2140a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:02.174307 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a00aae-5f18-4005-8dc8-bd4b2e2140a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:02.174596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a00aae-5f18-4005-8dc8-bd4b2e2140a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:02.174877 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a00aae-5f18-4005-8dc8-bd4b2e2140a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:02.176054 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a00aae-5f18-4005-8dc8-bd4b2e2140a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:02.179269 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a00aae-5f18-4005-8dc8-bd4b2e2140a5 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:02.217360 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 892/1803] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:01 2022] POST /identity/v3/users => generated 279 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:02.224476 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a00aae-5f18-4005-8dc8-bd4b2e2140a5 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:02.227720 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-04a00aae-5f18-4005-8dc8-bd4b2e2140a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:02.239692 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d4a0bd-9016-426c-8297-3d68fe36a82e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:02.240679 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a00aae-5f18-4005-8dc8-bd4b2e2140a5 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:02.247752 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2857fa-44dc-4634-b90f-9d8dfa476965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:02.248424 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2857fa-44dc-4634-b90f-9d8dfa476965 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:02.248770 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2857fa-44dc-4634-b90f-9d8dfa476965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:02.249132 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2857fa-44dc-4634-b90f-9d8dfa476965 admin admin] PATH_INFO: `/v3/projects/5b25797600d14e538e35108ef00ed07a` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:02.251355 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2857fa-44dc-4634-b90f-9d8dfa476965 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b25797600d14e538e35108ef00ed07a)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:02.252273 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d94ae-0d77-4543-a8f3-9050781d3173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:02.253825 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d94ae-0d77-4543-a8f3-9050781d3173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:02.254790 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2857fa-44dc-4634-b90f-9d8dfa476965 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:02.267244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d94ae-0d77-4543-a8f3-9050781d3173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:02.267968 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d94ae-0d77-4543-a8f3-9050781d3173 admin admin] PATH_INFO: `/v3/users/a7a30ae6358f48ac8559388ab864e580` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:02.275618 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8d94ae-0d77-4543-a8f3-9050781d3173 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7a30ae6358f48ac8559388ab864e580)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:02.280312 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8d94ae-0d77-4543-a8f3-9050781d3173 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:02.360938 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-8b2857fa-44dc-4634-b90f-9d8dfa476965 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b25797600d14e538e35108ef00ed07a', 'request_id': 'req-8b2857fa-44dc-4634-b90f-9d8dfa476965'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:02.376525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-eb8d94ae-0d77-4543-a8f3-9050781d3173 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7a30ae6358f48ac8559388ab864e580'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:02.379327 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-eb8d94ae-0d77-4543-a8f3-9050781d3173 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7a30ae6358f48ac8559388ab864e580'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:02.380734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-eb8d94ae-0d77-4543-a8f3-9050781d3173 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7a30ae6358f48ac8559388ab864e580'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:02.389808 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-8b2857fa-44dc-4634-b90f-9d8dfa476965 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b25797600d14e538e35108ef00ed07a', 'request_id': 'req-8b2857fa-44dc-4634-b90f-9d8dfa476965'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:02.396783 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-8b2857fa-44dc-4634-b90f-9d8dfa476965 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b25797600d14e538e35108ef00ed07a', 'request_id': 'req-8b2857fa-44dc-4634-b90f-9d8dfa476965'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:02.401747 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-eb8d94ae-0d77-4543-a8f3-9050781d3173 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7a30ae6358f48ac8559388ab864e580'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:02.412456 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-eb8d94ae-0d77-4543-a8f3-9050781d3173 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7a30ae6358f48ac8559388ab864e580'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:02.412846 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-8b2857fa-44dc-4634-b90f-9d8dfa476965 admin admin] The token cache is being invalidate because project 5b25797600d14e538e35108ef00ed07a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:02.413135 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-8b2857fa-44dc-4634-b90f-9d8dfa476965 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:02.414530 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-8b2857fa-44dc-4634-b90f-9d8dfa476965 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:02.420147 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 893/1804] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:02 2022] DELETE /identity/v3/projects/5b25797600d14e538e35108ef00ed07a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:02.420147 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2857fa-44dc-4634-b90f-9d8dfa476965 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:02.439585 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 912/1805] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:02 2022] DELETE /identity/v3/users/a7a30ae6358f48ac8559388ab864e580 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:02.439585 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e865b-e90a-451e-90e4-330f53dc80a3 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:02.439585 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e865b-e90a-451e-90e4-330f53dc80a3 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:02.440324 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e865b-e90a-451e-90e4-330f53dc80a3 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:02.442702 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 913/1806] 10.222.0.46 () {54 vars in 921 bytes} [Wed Jul 27 04:22:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:22:02.443615 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-8b2857fa-44dc-4634-b90f-9d8dfa476965 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:02.448534 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e865b-e90a-451e-90e4-330f53dc80a3 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:02.450472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-550e865b-e90a-451e-90e4-330f53dc80a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:02.538109 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:02.548904 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:02.580719 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e865b-e90a-451e-90e4-330f53dc80a3 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:02.600692 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4d6fa-9b0f-4d8a-9d66-b88b9c6b95c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:02.605051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4d6fa-9b0f-4d8a-9d66-b88b9c6b95c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:02.605436 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4d6fa-9b0f-4d8a-9d66-b88b9c6b95c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:02.605721 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4d6fa-9b0f-4d8a-9d66-b88b9c6b95c9 admin admin] PATH_INFO: `/v3/projects/04b921e9d954424f9c1822e2fc77e36e` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:02.606613 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2857fa-44dc-4634-b90f-9d8dfa476965 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:02.607659 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb4d6fa-9b0f-4d8a-9d66-b88b9c6b95c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04b921e9d954424f9c1822e2fc77e36e)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:02.615867 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1035c17-6c68-4896-ae00-8eb79c3a8245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:02.616528 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1035c17-6c68-4896-ae00-8eb79c3a8245 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:02.616807 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1035c17-6c68-4896-ae00-8eb79c3a8245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:02.617085 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1035c17-6c68-4896-ae00-8eb79c3a8245 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:02.617197 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb4d6fa-9b0f-4d8a-9d66-b88b9c6b95c9 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:02.617901 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1035c17-6c68-4896-ae00-8eb79c3a8245 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:02.619745 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1035c17-6c68-4896-ae00-8eb79c3a8245 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:02.638032 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 894/1807] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:02 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:02.638032 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8b925-b4e8-4f39-9db9-0b91f1627adf None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:02.638032 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8b925-b4e8-4f39-9db9-0b91f1627adf None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:02.638032 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8b925-b4e8-4f39-9db9-0b91f1627adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:02.679482 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-66e8b925-b4e8-4f39-9db9-0b91f1627adf None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:02.681092 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-66e8b925-b4e8-4f39-9db9-0b91f1627adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:02.693129 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 895/1808] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:02.695270 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1761aa2-1fbc-468c-995c-a2f5a83c273a None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:02.695571 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1761aa2-1fbc-468c-995c-a2f5a83c273a None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:02.695898 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1761aa2-1fbc-468c-995c-a2f5a83c273a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:02.728198 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-e1761aa2-1fbc-468c-995c-a2f5a83c273a None None] MFA Rules not processed for user `9dc9ea8f89c14a2a8e3272f750647dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:02.730244 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e1761aa2-1fbc-468c-995c-a2f5a83c273a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:02.762152 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fbb4d6fa-9b0f-4d8a-9d66-b88b9c6b95c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04b921e9d954424f9c1822e2fc77e36e', 'request_id': 'req-fbb4d6fa-9b0f-4d8a-9d66-b88b9c6b95c9'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:02.771563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fbb4d6fa-9b0f-4d8a-9d66-b88b9c6b95c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04b921e9d954424f9c1822e2fc77e36e', 'request_id': 'req-fbb4d6fa-9b0f-4d8a-9d66-b88b9c6b95c9'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:02.786306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fbb4d6fa-9b0f-4d8a-9d66-b88b9c6b95c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04b921e9d954424f9c1822e2fc77e36e', 'request_id': 'req-fbb4d6fa-9b0f-4d8a-9d66-b88b9c6b95c9'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:02.814131 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fbb4d6fa-9b0f-4d8a-9d66-b88b9c6b95c9 admin admin] The token cache is being invalidate because project 04b921e9d954424f9c1822e2fc77e36e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:02.814564 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fbb4d6fa-9b0f-4d8a-9d66-b88b9c6b95c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:02.815854 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fbb4d6fa-9b0f-4d8a-9d66-b88b9c6b95c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:02.820297 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 914/1809] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:02 2022] DELETE /identity/v3/projects/04b921e9d954424f9c1822e2fc77e36e => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:02.823124 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4d6fa-9b0f-4d8a-9d66-b88b9c6b95c9 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:02.826814 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-fbb4d6fa-9b0f-4d8a-9d66-b88b9c6b95c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:02.876939 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 896/1810] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Jul 27 04:22:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:02.879200 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c1025-a13d-4f24-8b8b-29805a41c3fd None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:02.879432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c1025-a13d-4f24-8b8b-29805a41c3fd None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:02.881135 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c1025-a13d-4f24-8b8b-29805a41c3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:02.927096 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-e44c1025-a13d-4f24-8b8b-29805a41c3fd None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:02.928329 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e44c1025-a13d-4f24-8b8b-29805a41c3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:02.972888 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4d6fa-9b0f-4d8a-9d66-b88b9c6b95c9 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:02.980776 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca872685-6df0-48d0-bc58-2215e2d3c960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:02.982176 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca872685-6df0-48d0-bc58-2215e2d3c960 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:02.982635 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca872685-6df0-48d0-bc58-2215e2d3c960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:02.983075 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca872685-6df0-48d0-bc58-2215e2d3c960 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:02.984101 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca872685-6df0-48d0-bc58-2215e2d3c960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:02.986887 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca872685-6df0-48d0-bc58-2215e2d3c960 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:03.029764 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 915/1811] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:02 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:03.033467 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2efb5d5-3f8a-48b7-b614-b048cd9ebb5e None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:03.036973 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 897/1812] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:03.038246 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c1025-a13d-4f24-8b8b-29805a41c3fd None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:03.040222 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e44c1025-a13d-4f24-8b8b-29805a41c3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:03.048360 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2efb5d5-3f8a-48b7-b614-b048cd9ebb5e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:03.049959 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2efb5d5-3f8a-48b7-b614-b048cd9ebb5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:03.066749 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c1025-a13d-4f24-8b8b-29805a41c3fd None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:03.079507 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17a6737-261d-4f13-b7a8-17d59ee50cb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:03.080662 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a6737-261d-4f13-b7a8-17d59ee50cb9 service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:03.081080 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a6737-261d-4f13-b7a8-17d59ee50cb9 service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:03.090593 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a6737-261d-4f13-b7a8-17d59ee50cb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:03.110892 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17a6737-261d-4f13-b7a8-17d59ee50cb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:03.113830 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17a6737-261d-4f13-b7a8-17d59ee50cb9 service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:03.115342 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-b2efb5d5-3f8a-48b7-b614-b048cd9ebb5e None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:03.116617 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b2efb5d5-3f8a-48b7-b614-b048cd9ebb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:03.138315 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 916/1813] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:03.139433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2efb5d5-3f8a-48b7-b614-b048cd9ebb5e None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:03.140937 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b2efb5d5-3f8a-48b7-b614-b048cd9ebb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:03.149897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 898/1814] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:03.151034 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17a6737-261d-4f13-b7a8-17d59ee50cb9 service nova] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:03.156551 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2efb5d5-3f8a-48b7-b614-b048cd9ebb5e None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:03.162339 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0798eea2-25c9-46f7-95b5-c0ebcaa9f971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:03.163437 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798eea2-25c9-46f7-95b5-c0ebcaa9f971 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:03.163885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798eea2-25c9-46f7-95b5-c0ebcaa9f971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:03.164312 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798eea2-25c9-46f7-95b5-c0ebcaa9f971 admin admin] PATH_INFO: `/v3/projects/fac2034a19a94baea512aad71fe1a111` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:03.167533 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0798eea2-25c9-46f7-95b5-c0ebcaa9f971 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fac2034a19a94baea512aad71fe1a111)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:03.170065 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0798eea2-25c9-46f7-95b5-c0ebcaa9f971 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:03.181151 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17a6737-261d-4f13-b7a8-17d59ee50cb9 service nova] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:03.199213 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216221e9-7233-4012-a29d-c1ef4e670698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:03.200066 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216221e9-7233-4012-a29d-c1ef4e670698 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:03.200449 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216221e9-7233-4012-a29d-c1ef4e670698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:03.200833 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216221e9-7233-4012-a29d-c1ef4e670698 admin admin] PATH_INFO: `/v3/projects/288d611a37a1446dab8e4465d5c90ca6/users/93c2e138a825454999ae5caf8e164e87/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:03.241404 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0798eea2-25c9-46f7-95b5-c0ebcaa9f971 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fac2034a19a94baea512aad71fe1a111', 'request_id': 'req-0798eea2-25c9-46f7-95b5-c0ebcaa9f971'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:03.242098 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216221e9-7233-4012-a29d-c1ef4e670698 admin admin] RBAC: Authorizing `identity:create_grant(project_id=288d611a37a1446dab8e4465d5c90ca6, user_id=93c2e138a825454999ae5caf8e164e87, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:03.245466 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216221e9-7233-4012-a29d-c1ef4e670698 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:03.250945 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0798eea2-25c9-46f7-95b5-c0ebcaa9f971 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fac2034a19a94baea512aad71fe1a111', 'request_id': 'req-0798eea2-25c9-46f7-95b5-c0ebcaa9f971'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:03.253513 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:03.271140 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0798eea2-25c9-46f7-95b5-c0ebcaa9f971 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fac2034a19a94baea512aad71fe1a111', 'request_id': 'req-0798eea2-25c9-46f7-95b5-c0ebcaa9f971'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:03.284381 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0798eea2-25c9-46f7-95b5-c0ebcaa9f971 admin admin] The token cache is being invalidate because project fac2034a19a94baea512aad71fe1a111 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:03.284673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0798eea2-25c9-46f7-95b5-c0ebcaa9f971 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:03.295633 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 899/1815] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:03 2022] PUT /identity/v3/projects/288d611a37a1446dab8e4465d5c90ca6/users/93c2e138a825454999ae5caf8e164e87/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:03.295843 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0798eea2-25c9-46f7-95b5-c0ebcaa9f971 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:03.295950 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216221e9-7233-4012-a29d-c1ef4e670698 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:03.296038 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 917/1816] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:03 2022] DELETE /identity/v3/projects/fac2034a19a94baea512aad71fe1a111 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:03.296038 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6d53f-4ce7-426b-a01e-2ab3d3feb125 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:03.296038 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6d53f-4ce7-426b-a01e-2ab3d3feb125 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:03.296038 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6d53f-4ce7-426b-a01e-2ab3d3feb125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:03.302609 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-216221e9-7233-4012-a29d-c1ef4e670698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:03.316707 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-78c6d53f-4ce7-426b-a01e-2ab3d3feb125 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:03.317917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-78c6d53f-4ce7-426b-a01e-2ab3d3feb125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:03.340756 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:03.381177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216221e9-7233-4012-a29d-c1ef4e670698 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:03.387331 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c587e68-61a4-491b-ba70-2a96b3574947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:03.387973 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c587e68-61a4-491b-ba70-2a96b3574947 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:03.388257 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c587e68-61a4-491b-ba70-2a96b3574947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:03.388539 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c587e68-61a4-491b-ba70-2a96b3574947 admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:03.389397 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c587e68-61a4-491b-ba70-2a96b3574947 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:22:03.389714 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c587e68-61a4-491b-ba70-2a96b3574947 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:03.391403 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c587e68-61a4-491b-ba70-2a96b3574947 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:03.404563 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 900/1817] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:22:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:22:03.407926 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273687c4-187b-4314-9487-caed7607c5af None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:03.408425 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273687c4-187b-4314-9487-caed7607c5af None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:03.408878 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273687c4-187b-4314-9487-caed7607c5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:03.420094 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 918/1818] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:03.421242 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd5f7f-3668-4880-a892-c4ed383eb80d None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:03.421530 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd5f7f-3668-4880-a892-c4ed383eb80d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:03.422122 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd5f7f-3668-4880-a892-c4ed383eb80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:03.451724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-273687c4-187b-4314-9487-caed7607c5af None None] MFA Rules not processed for user `93c2e138a825454999ae5caf8e164e87`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:03.452969 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-273687c4-187b-4314-9487-caed7607c5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:03.454868 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-8cfd5f7f-3668-4880-a892-c4ed383eb80d None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:03.456205 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8cfd5f7f-3668-4880-a892-c4ed383eb80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:03.482463 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 919/1819] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:03.490828 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd5f7f-3668-4880-a892-c4ed383eb80d None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:03.504708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd5f7f-3668-4880-a892-c4ed383eb80d None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:03.510415 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ffae91-f50e-4da4-a95b-e82235866763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:03.511045 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffae91-f50e-4da4-a95b-e82235866763 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:03.511328 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffae91-f50e-4da4-a95b-e82235866763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:03.511606 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffae91-f50e-4da4-a95b-e82235866763 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:03.512292 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ffae91-f50e-4da4-a95b-e82235866763 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:03.514231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ffae91-f50e-4da4-a95b-e82235866763 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:03.535271 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 901/1820] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:03 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:03.536100 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273687c4-187b-4314-9487-caed7607c5af None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:03.546373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 920/1821] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:03 2022] POST /identity/v3/projects => generated 393 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:03.547729 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad4e69-0852-419a-8358-3a935b1b6902 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:03.548015 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad4e69-0852-419a-8358-3a935b1b6902 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:03.548412 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad4e69-0852-419a-8358-3a935b1b6902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:03.549746 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273687c4-187b-4314-9487-caed7607c5af None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:03.569165 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-4cad4e69-0852-419a-8358-3a935b1b6902 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:03.570753 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-4cad4e69-0852-419a-8358-3a935b1b6902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:03.632549 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2af27-e155-48a0-8ade-ae803e1592fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:03.634328 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2af27-e155-48a0-8ade-ae803e1592fe admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:03.636585 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2af27-e155-48a0-8ade-ae803e1592fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:03.637608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2af27-e155-48a0-8ade-ae803e1592fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:03.638614 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2af27-e155-48a0-8ade-ae803e1592fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:22:03.640131 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2af27-e155-48a0-8ade-ae803e1592fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:03.643418 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2af27-e155-48a0-8ade-ae803e1592fe admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:03.655006 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 921/1822] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:03.656093 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad4e69-0852-419a-8358-3a935b1b6902 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:03.657007 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 902/1823] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:22:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:22:03.657700 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2af27-e155-48a0-8ade-ae803e1592fe admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:03.662878 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-4cad4e69-0852-419a-8358-3a935b1b6902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:03.669239 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2af27-e155-48a0-8ade-ae803e1592fe admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:03.675059 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad4e69-0852-419a-8358-3a935b1b6902 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:03.675295 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb81f4f-4169-4546-a4d7-1505a44ef298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:03.675921 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb81f4f-4169-4546-a4d7-1505a44ef298 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:03.676206 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb81f4f-4169-4546-a4d7-1505a44ef298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:03.676489 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb81f4f-4169-4546-a4d7-1505a44ef298 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:03.677209 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb81f4f-4169-4546-a4d7-1505a44ef298 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:03.679169 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb81f4f-4169-4546-a4d7-1505a44ef298 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:03.680451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c89e0-3f8a-4d89-ae76-3ea356f2d890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:03.681194 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c89e0-3f8a-4d89-ae76-3ea356f2d890 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:03.681989 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c89e0-3f8a-4d89-ae76-3ea356f2d890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:03.681989 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c89e0-3f8a-4d89-ae76-3ea356f2d890 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:03.683112 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c89e0-3f8a-4d89-ae76-3ea356f2d890 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:03.687977 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c89e0-3f8a-4d89-ae76-3ea356f2d890 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:03.727832 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 922/1824] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:03 2022] POST /identity/v3/projects => generated 355 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:03.729122 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c89e0-3f8a-4d89-ae76-3ea356f2d890 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:03.742338 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 903/1825] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:03 2022] POST /identity/v3/users => generated 297 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:03.743074 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb81f4f-4169-4546-a4d7-1505a44ef298 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:03.756202 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb81f4f-4169-4546-a4d7-1505a44ef298 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:03.800567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:03.813746 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:03.825594 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c89e0-3f8a-4d89-ae76-3ea356f2d890 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:03.832156 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db3775-c1aa-46c9-a617-42c001c8dfc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:03.832951 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db3775-c1aa-46c9-a617-42c001c8dfc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:03.833309 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db3775-c1aa-46c9-a617-42c001c8dfc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:03.833742 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db3775-c1aa-46c9-a617-42c001c8dfc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:03.834794 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db3775-c1aa-46c9-a617-42c001c8dfc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:22:03.835250 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db3775-c1aa-46c9-a617-42c001c8dfc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:03.837058 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db3775-c1aa-46c9-a617-42c001c8dfc3 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:03.862515 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 923/1826] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:22:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:22:03.863917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db3775-c1aa-46c9-a617-42c001c8dfc3 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:03.876956 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db3775-c1aa-46c9-a617-42c001c8dfc3 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:03.882945 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c2847-5f77-4454-b880-884ad780b011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:03.883592 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c2847-5f77-4454-b880-884ad780b011 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:03.883871 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c2847-5f77-4454-b880-884ad780b011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:03.884203 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c2847-5f77-4454-b880-884ad780b011 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:03.885113 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52c2847-5f77-4454-b880-884ad780b011 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:03.888407 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52c2847-5f77-4454-b880-884ad780b011 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:03.894786 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0069bf-caa0-4af2-a2d0-bef5d05c1d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:03.895457 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0069bf-caa0-4af2-a2d0-bef5d05c1d92 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:03.895748 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0069bf-caa0-4af2-a2d0-bef5d05c1d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:03.896089 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0069bf-caa0-4af2-a2d0-bef5d05c1d92 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:03.896721 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0069bf-caa0-4af2-a2d0-bef5d05c1d92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:03.903005 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0069bf-caa0-4af2-a2d0-bef5d05c1d92 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:03.932743 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 924/1827] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:03 2022] POST /identity/v3/users => generated 284 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:03.934163 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0069bf-caa0-4af2-a2d0-bef5d05c1d92 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:03.965655 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0069bf-caa0-4af2-a2d0-bef5d05c1d92 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:03.971585 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 904/1828] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:03 2022] POST /identity/v3/projects => generated 365 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:03.971585 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c2847-5f77-4454-b880-884ad780b011 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:03.971825 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9377c-da3e-43ef-8382-a4d62e4927ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:03.971825 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9377c-da3e-43ef-8382-a4d62e4927ce admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:03.971825 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9377c-da3e-43ef-8382-a4d62e4927ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:03.971825 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9377c-da3e-43ef-8382-a4d62e4927ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:03.972518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd9377c-da3e-43ef-8382-a4d62e4927ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:03.974176 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd9377c-da3e-43ef-8382-a4d62e4927ce admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:03.985679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 925/1829] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:03 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:03.986546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9377c-da3e-43ef-8382-a4d62e4927ce admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:04.000870 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9377c-da3e-43ef-8382-a4d62e4927ce admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:04.071038 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c2847-5f77-4454-b880-884ad780b011 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:04.086135 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b521d8-26b8-42b5-9444-5a7a0eb2e45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:04.087339 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b521d8-26b8-42b5-9444-5a7a0eb2e45f admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:04.088408 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b521d8-26b8-42b5-9444-5a7a0eb2e45f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:04.088940 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b521d8-26b8-42b5-9444-5a7a0eb2e45f admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:04.089868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b521d8-26b8-42b5-9444-5a7a0eb2e45f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:04.093404 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b521d8-26b8-42b5-9444-5a7a0eb2e45f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:04.093676 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba38dc2-aaac-4af6-a3f8-816b9040f968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:04.094697 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba38dc2-aaac-4af6-a3f8-816b9040f968 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:04.095150 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba38dc2-aaac-4af6-a3f8-816b9040f968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:04.095624 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba38dc2-aaac-4af6-a3f8-816b9040f968 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:04.116698 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba38dc2-aaac-4af6-a3f8-816b9040f968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:04.118395 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba38dc2-aaac-4af6-a3f8-816b9040f968 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:04.149731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 926/1830] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:03 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:04.154472 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 905/1831] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:03 2022] POST /identity/v3/projects => generated 407 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:04.155488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b521d8-26b8-42b5-9444-5a7a0eb2e45f admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:04.161552 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba38dc2-aaac-4af6-a3f8-816b9040f968 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:04.174620 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba38dc2-aaac-4af6-a3f8-816b9040f968 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:04.252261 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b521d8-26b8-42b5-9444-5a7a0eb2e45f admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:04.253206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6501b0-6d6c-4eff-9cf6-13c4a644cfe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:04.254144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6501b0-6d6c-4eff-9cf6-13c4a644cfe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:04.254451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6501b0-6d6c-4eff-9cf6-13c4a644cfe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:04.255625 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6501b0-6d6c-4eff-9cf6-13c4a644cfe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:04.261436 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6501b0-6d6c-4eff-9cf6-13c4a644cfe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:04.262743 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6501b0-6d6c-4eff-9cf6-13c4a644cfe9 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:04.262925 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d7abb-5fa6-4c56-8c15-91a934687a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:04.264159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d7abb-5fa6-4c56-8c15-91a934687a5c admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:04.264645 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d7abb-5fa6-4c56-8c15-91a934687a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:04.278382 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d7abb-5fa6-4c56-8c15-91a934687a5c admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:04.279770 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0d7abb-5fa6-4c56-8c15-91a934687a5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:04.284212 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0d7abb-5fa6-4c56-8c15-91a934687a5c admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:04.297672 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 927/1832] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:04.298693 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6501b0-6d6c-4eff-9cf6-13c4a644cfe9 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:04.309085 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6501b0-6d6c-4eff-9cf6-13c4a644cfe9 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:04.317573 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcbc564-9273-425c-a1cc-8b0b7f61bcaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:04.318594 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbc564-9273-425c-a1cc-8b0b7f61bcaa admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:04.319143 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbc564-9273-425c-a1cc-8b0b7f61bcaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:04.319775 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbc564-9273-425c-a1cc-8b0b7f61bcaa admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:04.320624 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcbc564-9273-425c-a1cc-8b0b7f61bcaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:04.323810 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcbc564-9273-425c-a1cc-8b0b7f61bcaa admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:04.336653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 906/1833] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:04 2022] POST /identity/v3/users => generated 283 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:04.339097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d7abb-5fa6-4c56-8c15-91a934687a5c admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:04.352345 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d7abb-5fa6-4c56-8c15-91a934687a5c admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:04.358703 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24892daf-fa9d-4109-8d43-511a4440ae4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:04.359657 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24892daf-fa9d-4109-8d43-511a4440ae4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:04.360048 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24892daf-fa9d-4109-8d43-511a4440ae4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:04.360479 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24892daf-fa9d-4109-8d43-511a4440ae4c admin admin] PATH_INFO: `/v3/projects/f05c0f7bf505463dae8da1318f2a1d6a/users/f373ca22369b4b3fb77d7a45b66e9a82/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:04.370684 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 928/1834] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:04 2022] POST /identity/v3/users => generated 304 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:04.371963 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcbc564-9273-425c-a1cc-8b0b7f61bcaa admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:04.389348 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcbc564-9273-425c-a1cc-8b0b7f61bcaa admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:04.395913 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24892daf-fa9d-4109-8d43-511a4440ae4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f05c0f7bf505463dae8da1318f2a1d6a, user_id=f373ca22369b4b3fb77d7a45b66e9a82, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:04.396967 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41293dff-1ba0-46a2-bd27-1b6ec4a4ab36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:04.397843 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41293dff-1ba0-46a2-bd27-1b6ec4a4ab36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:04.399970 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24892daf-fa9d-4109-8d43-511a4440ae4c admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:04.401248 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41293dff-1ba0-46a2-bd27-1b6ec4a4ab36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:04.401688 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41293dff-1ba0-46a2-bd27-1b6ec4a4ab36 admin admin] PATH_INFO: `/v3/projects/14f8f6245b4244998d4a30c5717a1480/users/db44c99bb91a4da1ab75f9e7194be739/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:04.420339 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:04.422578 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 907/1835] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:04 2022] PUT /identity/v3/projects/f05c0f7bf505463dae8da1318f2a1d6a/users/f373ca22369b4b3fb77d7a45b66e9a82/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:04.423848 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24892daf-fa9d-4109-8d43-511a4440ae4c admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:04.425495 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41293dff-1ba0-46a2-bd27-1b6ec4a4ab36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14f8f6245b4244998d4a30c5717a1480, user_id=db44c99bb91a4da1ab75f9e7194be739, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:04.430104 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41293dff-1ba0-46a2-bd27-1b6ec4a4ab36 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:04.432312 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:04.472603 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 929/1836] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:04 2022] PUT /identity/v3/projects/14f8f6245b4244998d4a30c5717a1480/users/db44c99bb91a4da1ab75f9e7194be739/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:04.474364 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41293dff-1ba0-46a2-bd27-1b6ec4a4ab36 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:04.556343 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24892daf-fa9d-4109-8d43-511a4440ae4c admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:04.559277 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41293dff-1ba0-46a2-bd27-1b6ec4a4ab36 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:04.566663 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc22dc5-93c2-4665-9925-325a6ebef2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:04.567314 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc22dc5-93c2-4665-9925-325a6ebef2d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:04.567599 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc22dc5-93c2-4665-9925-325a6ebef2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:04.567878 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc22dc5-93c2-4665-9925-325a6ebef2d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:04.568593 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc22dc5-93c2-4665-9925-325a6ebef2d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:04.570077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be234ed0-8ab2-41ac-92b3-c020f0952dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:04.570511 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc22dc5-93c2-4665-9925-325a6ebef2d2 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:04.570990 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be234ed0-8ab2-41ac-92b3-c020f0952dda admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:04.571367 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be234ed0-8ab2-41ac-92b3-c020f0952dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:04.571745 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be234ed0-8ab2-41ac-92b3-c020f0952dda admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:04.573191 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be234ed0-8ab2-41ac-92b3-c020f0952dda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:04.575320 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be234ed0-8ab2-41ac-92b3-c020f0952dda admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:04.587402 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 930/1837] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:04.589413 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc22dc5-93c2-4665-9925-325a6ebef2d2 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:04.597730 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 908/1838] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:04.599196 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ee2ff-b753-4d1a-9029-fae48c9e3a79 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:04.599488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ee2ff-b753-4d1a-9029-fae48c9e3a79 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:04.599767 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ee2ff-b753-4d1a-9029-fae48c9e3a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:04.601843 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc22dc5-93c2-4665-9925-325a6ebef2d2 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:04.608491 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f522c2a2-ca30-49ed-8151-e488eca6cfc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:04.609546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522c2a2-ca30-49ed-8151-e488eca6cfc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:04.609928 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522c2a2-ca30-49ed-8151-e488eca6cfc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:04.610338 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522c2a2-ca30-49ed-8151-e488eca6cfc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:04.611112 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f522c2a2-ca30-49ed-8151-e488eca6cfc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:04.612782 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f522c2a2-ca30-49ed-8151-e488eca6cfc0 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:04.623474 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 931/1839] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:04.624362 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f522c2a2-ca30-49ed-8151-e488eca6cfc0 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:04.631908 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-563ee2ff-b753-4d1a-9029-fae48c9e3a79 None None] MFA Rules not processed for user `db44c99bb91a4da1ab75f9e7194be739`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:04.638068 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-563ee2ff-b753-4d1a-9029-fae48c9e3a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:04.638235 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f522c2a2-ca30-49ed-8151-e488eca6cfc0 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:04.642719 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b3a02-5484-49d8-ad16-8c9c1d58b1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:04.643436 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b3a02-5484-49d8-ad16-8c9c1d58b1c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:04.643794 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b3a02-5484-49d8-ad16-8c9c1d58b1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:04.644144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b3a02-5484-49d8-ad16-8c9c1d58b1c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:04.644929 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b3a02-5484-49d8-ad16-8c9c1d58b1c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:04.646644 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b3a02-5484-49d8-ad16-8c9c1d58b1c0 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:04.660130 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 932/1840] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:04.661091 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b3a02-5484-49d8-ad16-8c9c1d58b1c0 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:04.672649 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b3a02-5484-49d8-ad16-8c9c1d58b1c0 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:04.678382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6321c-0b2c-4802-becc-1be6248a7d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:04.678992 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6321c-0b2c-4802-becc-1be6248a7d7e admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:04.679448 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6321c-0b2c-4802-becc-1be6248a7d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:04.679806 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6321c-0b2c-4802-becc-1be6248a7d7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:04.680708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a6321c-0b2c-4802-becc-1be6248a7d7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:04.682851 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a6321c-0b2c-4802-becc-1be6248a7d7e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:04.696140 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 933/1841] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:04.696688 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6321c-0b2c-4802-becc-1be6248a7d7e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:04.708731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6321c-0b2c-4802-becc-1be6248a7d7e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:04.718546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72caafb0-c648-4c90-8bca-373fa93710b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:04.719423 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72caafb0-c648-4c90-8bca-373fa93710b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:04.719830 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72caafb0-c648-4c90-8bca-373fa93710b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:04.720363 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72caafb0-c648-4c90-8bca-373fa93710b1 admin admin] PATH_INFO: `/v3/projects/f05c0f7bf505463dae8da1318f2a1d6a/users/f373ca22369b4b3fb77d7a45b66e9a82/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:04.722826 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72caafb0-c648-4c90-8bca-373fa93710b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f05c0f7bf505463dae8da1318f2a1d6a, user_id=f373ca22369b4b3fb77d7a45b66e9a82, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:04.730912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72caafb0-c648-4c90-8bca-373fa93710b1 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:04.738852 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 909/1842] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:04.743115 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ee2ff-b753-4d1a-9029-fae48c9e3a79 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:04.743689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 934/1843] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:04 2022] PUT /identity/v3/projects/f05c0f7bf505463dae8da1318f2a1d6a/users/f373ca22369b4b3fb77d7a45b66e9a82/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:04.744558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72caafb0-c648-4c90-8bca-373fa93710b1 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:04.759457 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ee2ff-b753-4d1a-9029-fae48c9e3a79 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:04.821715 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72caafb0-c648-4c90-8bca-373fa93710b1 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:04.827928 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8638b-ad27-49b2-a491-c3e679c45a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:04.828764 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8638b-ad27-49b2-a491-c3e679c45a19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:04.829157 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8638b-ad27-49b2-a491-c3e679c45a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:04.829537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8638b-ad27-49b2-a491-c3e679c45a19 admin admin] PATH_INFO: `/v3/projects/734a7a0ca0ed466f8c4ff18a480cdfd9/users/7912374aed7c4ea8922a8eebb9a3321e/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:04.846248 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f8638b-ad27-49b2-a491-c3e679c45a19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=734a7a0ca0ed466f8c4ff18a480cdfd9, user_id=7912374aed7c4ea8922a8eebb9a3321e, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:04.849043 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f8638b-ad27-49b2-a491-c3e679c45a19 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:04.858831 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74100e02-ebba-4121-821f-5ca93ba3b2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:04.859701 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74100e02-ebba-4121-821f-5ca93ba3b2b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:04.859949 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74100e02-ebba-4121-821f-5ca93ba3b2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:04.860222 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74100e02-ebba-4121-821f-5ca93ba3b2b3 admin admin] PATH_INFO: `/v3/projects/5c69dc7607c94b58b8335101487bb6d8/users/303ffab8fb7b4b30a1c7e60dd4869f21/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:04.871020 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 935/1844] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:04 2022] PUT /identity/v3/projects/734a7a0ca0ed466f8c4ff18a480cdfd9/users/7912374aed7c4ea8922a8eebb9a3321e/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:04.871020 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8638b-ad27-49b2-a491-c3e679c45a19 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:04.879295 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74100e02-ebba-4121-821f-5ca93ba3b2b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c69dc7607c94b58b8335101487bb6d8, user_id=303ffab8fb7b4b30a1c7e60dd4869f21, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:04.882611 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74100e02-ebba-4121-821f-5ca93ba3b2b3 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:04.902533 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 910/1845] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:04 2022] PUT /identity/v3/projects/5c69dc7607c94b58b8335101487bb6d8/users/303ffab8fb7b4b30a1c7e60dd4869f21/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:04.911835 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f08e2-0020-42d3-97f4-a0bc31d959fd None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:04.911835 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f08e2-0020-42d3-97f4-a0bc31d959fd None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:04.911835 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f08e2-0020-42d3-97f4-a0bc31d959fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:04.944891 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:04.952159 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:04.953417 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-3d1f08e2-0020-42d3-97f4-a0bc31d959fd None None] MFA Rules not processed for user `f373ca22369b4b3fb77d7a45b66e9a82`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:04.958735 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3d1f08e2-0020-42d3-97f4-a0bc31d959fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:05.036171 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8638b-ad27-49b2-a491-c3e679c45a19 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:05.093152 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 911/1846] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:04 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:05.095001 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31928f-dd37-4f22-9cb5-0eed4078b3ee None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:05.095447 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31928f-dd37-4f22-9cb5-0eed4078b3ee None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:05.095870 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31928f-dd37-4f22-9cb5-0eed4078b3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:05.145221 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-eb31928f-dd37-4f22-9cb5-0eed4078b3ee None None] MFA Rules not processed for user `7912374aed7c4ea8922a8eebb9a3321e`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:05.145221 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-eb31928f-dd37-4f22-9cb5-0eed4078b3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:05.200711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916ace2-6493-4fe9-9ce9-4dd8b95680eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:05.201760 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916ace2-6493-4fe9-9ce9-4dd8b95680eb admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:05.202221 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916ace2-6493-4fe9-9ce9-4dd8b95680eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:05.203023 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916ace2-6493-4fe9-9ce9-4dd8b95680eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:05.204057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b916ace2-6493-4fe9-9ce9-4dd8b95680eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:05.206935 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b916ace2-6493-4fe9-9ce9-4dd8b95680eb admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:05.253373 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 912/1847] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:05 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:05.253373 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ecc77-7f59-45e8-a236-2269311f16ab None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:05.253373 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ecc77-7f59-45e8-a236-2269311f16ab None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:05.253373 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ecc77-7f59-45e8-a236-2269311f16ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:05.267899 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 936/1848] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:04 2022] POST /identity/v3/projects => generated 393 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:05.268586 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916ace2-6493-4fe9-9ce9-4dd8b95680eb admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:05.274075 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b916ace2-6493-4fe9-9ce9-4dd8b95680eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:05.293590 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-551ecc77-7f59-45e8-a236-2269311f16ab None None] MFA Rules not processed for user `303ffab8fb7b4b30a1c7e60dd4869f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:05.295500 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-551ecc77-7f59-45e8-a236-2269311f16ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:05.363185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916ace2-6493-4fe9-9ce9-4dd8b95680eb admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:05.373402 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c14d0dc-ada1-427b-9dac-f087c384188f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:05.377312 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c14d0dc-ada1-427b-9dac-f087c384188f service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:05.377851 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c14d0dc-ada1-427b-9dac-f087c384188f service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:05.378388 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c14d0dc-ada1-427b-9dac-f087c384188f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:05.384487 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 913/1849] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:05 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:05.386592 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ecc77-7f59-45e8-a236-2269311f16ab None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:05.410625 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:05.474253 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c14d0dc-ada1-427b-9dac-f087c384188f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:05.475980 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c14d0dc-ada1-427b-9dac-f087c384188f service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:05.485548 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ecc77-7f59-45e8-a236-2269311f16ab None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:05.492029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 937/1850] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:05 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:05.493174 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c14d0dc-ada1-427b-9dac-f087c384188f service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:05.505810 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c14d0dc-ada1-427b-9dac-f087c384188f service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:05.513270 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb5ae6d-d192-4378-a8f8-2b77203a2722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:05.514304 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327727e-617c-4e8a-9d0c-679d166b58b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:05.514578 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5ae6d-d192-4378-a8f8-2b77203a2722 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:05.514981 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5ae6d-d192-4378-a8f8-2b77203a2722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:05.515234 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327727e-617c-4e8a-9d0c-679d166b58b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:05.515568 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5ae6d-d192-4378-a8f8-2b77203a2722 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:05.515736 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327727e-617c-4e8a-9d0c-679d166b58b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:05.515991 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327727e-617c-4e8a-9d0c-679d166b58b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:05.517523 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327727e-617c-4e8a-9d0c-679d166b58b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:05.517653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb5ae6d-d192-4378-a8f8-2b77203a2722 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:05.519099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327727e-617c-4e8a-9d0c-679d166b58b0 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:05.520066 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb5ae6d-d192-4378-a8f8-2b77203a2722 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:05.530877 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:05.560976 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 914/1851] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:05 2022] POST /identity/v3/projects => generated 363 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:05.560976 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327727e-617c-4e8a-9d0c-679d166b58b0 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:05.564161 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 938/1852] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:05 2022] POST /identity/v3/users => generated 303 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:05.566175 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb5ae6d-d192-4378-a8f8-2b77203a2722 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:05.581188 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb5ae6d-d192-4378-a8f8-2b77203a2722 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:05.700515 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327727e-617c-4e8a-9d0c-679d166b58b0 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:05.702946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8821148-5eff-4b31-b2c1-efa5887d6b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:05.703172 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8821148-5eff-4b31-b2c1-efa5887d6b6a admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:05.703575 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8821148-5eff-4b31-b2c1-efa5887d6b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:05.704511 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8821148-5eff-4b31-b2c1-efa5887d6b6a admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:05.705953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8821148-5eff-4b31-b2c1-efa5887d6b6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:05.708424 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8821148-5eff-4b31-b2c1-efa5887d6b6a admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:05.718244 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc4a822-a951-491c-b138-118de0326e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:05.718938 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4a822-a951-491c-b138-118de0326e35 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:05.725260 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4a822-a951-491c-b138-118de0326e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:05.725604 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4a822-a951-491c-b138-118de0326e35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:05.726648 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc4a822-a951-491c-b138-118de0326e35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:05.729515 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc4a822-a951-491c-b138-118de0326e35 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:05.740994 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 939/1853] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:05 2022] POST /identity/v3/users => generated 288 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:05.740994 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8821148-5eff-4b31-b2c1-efa5887d6b6a admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:05.756241 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8821148-5eff-4b31-b2c1-efa5887d6b6a admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:05.773605 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 915/1854] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:05 2022] POST /identity/v3/projects => generated 407 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:05.773605 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc4a822-a951-491c-b138-118de0326e35 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:05.780594 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a5db1-5b03-44ca-99db-6f102d7a299f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:05.781251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a5db1-5b03-44ca-99db-6f102d7a299f admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:05.781536 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a5db1-5b03-44ca-99db-6f102d7a299f admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:05.781813 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a5db1-5b03-44ca-99db-6f102d7a299f admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:05.782557 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a5db1-5b03-44ca-99db-6f102d7a299f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:05.784174 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a5db1-5b03-44ca-99db-6f102d7a299f admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:05.809078 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 940/1855] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:05 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:05.819995 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a5db1-5b03-44ca-99db-6f102d7a299f admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:05.831510 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a5db1-5b03-44ca-99db-6f102d7a299f admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:05.858044 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc4a822-a951-491c-b138-118de0326e35 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:05.863111 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340fe31-28dd-4192-bfd3-262722a81ae5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:05.864295 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340fe31-28dd-4192-bfd3-262722a81ae5 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:05.864653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340fe31-28dd-4192-bfd3-262722a81ae5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:05.865273 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340fe31-28dd-4192-bfd3-262722a81ae5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:05.912060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14f552-8312-4f7c-bf0b-17066950c3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:05.915323 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14f552-8312-4f7c-bf0b-17066950c3eb admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:05.915323 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14f552-8312-4f7c-bf0b-17066950c3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:05.915323 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14f552-8312-4f7c-bf0b-17066950c3eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:05.915323 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac14f552-8312-4f7c-bf0b-17066950c3eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:05.918622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac14f552-8312-4f7c-bf0b-17066950c3eb admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:05.926626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:05.935760 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 941/1856] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:05 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:05.935760 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14f552-8312-4f7c-bf0b-17066950c3eb admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:05.952365 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14f552-8312-4f7c-bf0b-17066950c3eb admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:05.954497 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a340fe31-28dd-4192-bfd3-262722a81ae5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:05.955962 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a340fe31-28dd-4192-bfd3-262722a81ae5 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:05.958740 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16ddba-a514-47d5-9e63-e1c61a9dd28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:05.959479 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16ddba-a514-47d5-9e63-e1c61a9dd28e admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:05.959829 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16ddba-a514-47d5-9e63-e1c61a9dd28e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:05.960177 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16ddba-a514-47d5-9e63-e1c61a9dd28e admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:05.961260 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa16ddba-a514-47d5-9e63-e1c61a9dd28e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:05.963598 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa16ddba-a514-47d5-9e63-e1c61a9dd28e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:05.966563 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 916/1857] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:05 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:05.967410 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340fe31-28dd-4192-bfd3-262722a81ae5 service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:05.984631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340fe31-28dd-4192-bfd3-262722a81ae5 service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:05.990681 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e7abd-c95b-4ce4-b9a1-360b3363ace1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:05.991374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e7abd-c95b-4ce4-b9a1-360b3363ace1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:05.991661 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e7abd-c95b-4ce4-b9a1-360b3363ace1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:05.991946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e7abd-c95b-4ce4-b9a1-360b3363ace1 admin admin] PATH_INFO: `/v3/projects/4598ce65aafd48499d3f790e58d531f9/users/226618f2857c4f44ba7228ebf1f5a2a6/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:05.992648 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 942/1858] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:05 2022] POST /identity/v3/users => generated 310 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:05.994993 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16ddba-a514-47d5-9e63-e1c61a9dd28e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:06.007073 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16ddba-a514-47d5-9e63-e1c61a9dd28e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:06.016034 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0e7abd-c95b-4ce4-b9a1-360b3363ace1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4598ce65aafd48499d3f790e58d531f9, user_id=226618f2857c4f44ba7228ebf1f5a2a6, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:06.019717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0e7abd-c95b-4ce4-b9a1-360b3363ace1 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:06.028959 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b3ee4-a4e6-4c9c-99af-167fb6b23173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:06.029944 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b3ee4-a4e6-4c9c-99af-167fb6b23173 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:06.030830 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b3ee4-a4e6-4c9c-99af-167fb6b23173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:06.031675 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b3ee4-a4e6-4c9c-99af-167fb6b23173 admin admin] PATH_INFO: `/v3/projects/a16db1aafd30434fbaf6d160e889b663/users/c0e539c97cc04d0da396b42dc28c9919/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:06.051711 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 917/1859] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:05 2022] PUT /identity/v3/projects/4598ce65aafd48499d3f790e58d531f9/users/226618f2857c4f44ba7228ebf1f5a2a6/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:06.052730 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e7abd-c95b-4ce4-b9a1-360b3363ace1 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:06.066642 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b3ee4-a4e6-4c9c-99af-167fb6b23173 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a16db1aafd30434fbaf6d160e889b663, user_id=c0e539c97cc04d0da396b42dc28c9919, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:06.070635 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b3ee4-a4e6-4c9c-99af-167fb6b23173 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:06.092412 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 943/1860] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:05 2022] PUT /identity/v3/projects/a16db1aafd30434fbaf6d160e889b663/users/c0e539c97cc04d0da396b42dc28c9919/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:06.093378 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b3ee4-a4e6-4c9c-99af-167fb6b23173 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:06.138367 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:INSERT=10,keystone:SELECT=90 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:06.234569 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b3ee4-a4e6-4c9c-99af-167fb6b23173 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:06.234705 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e7abd-c95b-4ce4-b9a1-360b3363ace1 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:06.242284 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df9678-1b73-4894-8ed0-3d4a2674849e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:06.243007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df9678-1b73-4894-8ed0-3d4a2674849e admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:06.243355 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df9678-1b73-4894-8ed0-3d4a2674849e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:06.243833 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df9678-1b73-4894-8ed0-3d4a2674849e admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:06.245159 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70df9678-1b73-4894-8ed0-3d4a2674849e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:06.247082 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70df9678-1b73-4894-8ed0-3d4a2674849e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:06.250598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253559ae-ac33-447c-843f-eda69c40de7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:06.251233 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253559ae-ac33-447c-843f-eda69c40de7d admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:06.251715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253559ae-ac33-447c-843f-eda69c40de7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:06.252003 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253559ae-ac33-447c-843f-eda69c40de7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:06.255166 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253559ae-ac33-447c-843f-eda69c40de7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:06.267857 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253559ae-ac33-447c-843f-eda69c40de7d admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:06.274308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 944/1861] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:06.275344 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df9678-1b73-4894-8ed0-3d4a2674849e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:06.286965 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df9678-1b73-4894-8ed0-3d4a2674849e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:06.287762 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 918/1862] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:06.288708 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253559ae-ac33-447c-843f-eda69c40de7d admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:06.297747 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49ea4a-e305-4037-9fc4-3c662e61031a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:06.298406 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49ea4a-e305-4037-9fc4-3c662e61031a admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:06.298689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49ea4a-e305-4037-9fc4-3c662e61031a admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:06.298964 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49ea4a-e305-4037-9fc4-3c662e61031a admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:06.299670 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49ea4a-e305-4037-9fc4-3c662e61031a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:06.301275 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49ea4a-e305-4037-9fc4-3c662e61031a admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:06.304075 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253559ae-ac33-447c-843f-eda69c40de7d admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:06.309868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a1718-4fc3-4fc5-afe5-490e2b0e228e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:06.310689 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a1718-4fc3-4fc5-afe5-490e2b0e228e admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:06.311062 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a1718-4fc3-4fc5-afe5-490e2b0e228e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:06.311403 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a1718-4fc3-4fc5-afe5-490e2b0e228e admin admin] PATH_INFO: `/v3/projects/4598ce65aafd48499d3f790e58d531f9/users/226618f2857c4f44ba7228ebf1f5a2a6/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:06.318284 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 945/1863] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:06.319867 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49ea4a-e305-4037-9fc4-3c662e61031a admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:06.327272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a1718-4fc3-4fc5-afe5-490e2b0e228e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4598ce65aafd48499d3f790e58d531f9, user_id=226618f2857c4f44ba7228ebf1f5a2a6, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:06.330480 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a1718-4fc3-4fc5-afe5-490e2b0e228e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:06.344175 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49ea4a-e305-4037-9fc4-3c662e61031a admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:06.347411 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 919/1864] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:06 2022] PUT /identity/v3/projects/4598ce65aafd48499d3f790e58d531f9/users/226618f2857c4f44ba7228ebf1f5a2a6/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:06.348317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a1718-4fc3-4fc5-afe5-490e2b0e228e admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:06.440055 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a1718-4fc3-4fc5-afe5-490e2b0e228e admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:06.455589 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92fac0-9f92-45be-b08d-7530173706d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:06.455589 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92fac0-9f92-45be-b08d-7530173706d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:06.455589 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92fac0-9f92-45be-b08d-7530173706d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:06.455589 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92fac0-9f92-45be-b08d-7530173706d0 admin admin] PATH_INFO: `/v3/projects/a16db1aafd30434fbaf6d160e889b663/users/c0e539c97cc04d0da396b42dc28c9919/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:06.459556 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b92fac0-9f92-45be-b08d-7530173706d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a16db1aafd30434fbaf6d160e889b663, user_id=c0e539c97cc04d0da396b42dc28c9919, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:06.467868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b92fac0-9f92-45be-b08d-7530173706d0 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:06.469224 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2626ff20-3725-4d2c-88de-635595ea48a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:06.470193 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626ff20-3725-4d2c-88de-635595ea48a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:06.470750 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626ff20-3725-4d2c-88de-635595ea48a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:06.471458 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626ff20-3725-4d2c-88de-635595ea48a4 admin admin] PATH_INFO: `/v3/projects/8940150716564661aec86b7727fdfcc3/users/6ab6637397064e1b82c9ecebe3d2aa01/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:06.481791 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 920/1865] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:06 2022] PUT /identity/v3/projects/a16db1aafd30434fbaf6d160e889b663/users/c0e539c97cc04d0da396b42dc28c9919/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:06.495038 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915c2a2f-3c99-49f4-8aef-c93af8798bcc None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:06.495474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915c2a2f-3c99-49f4-8aef-c93af8798bcc None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:06.496246 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915c2a2f-3c99-49f4-8aef-c93af8798bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:06.516045 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2626ff20-3725-4d2c-88de-635595ea48a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8940150716564661aec86b7727fdfcc3, user_id=6ab6637397064e1b82c9ecebe3d2aa01, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:06.523093 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2626ff20-3725-4d2c-88de-635595ea48a4 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:06.539989 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-915c2a2f-3c99-49f4-8aef-c93af8798bcc None None] MFA Rules not processed for user `226618f2857c4f44ba7228ebf1f5a2a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:06.542744 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-915c2a2f-3c99-49f4-8aef-c93af8798bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:06.553060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 946/1866] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:06 2022] PUT /identity/v3/projects/8940150716564661aec86b7727fdfcc3/users/6ab6637397064e1b82c9ecebe3d2aa01/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:06.554659 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80945f-5f02-4385-90af-c75abe65639b None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:06.555117 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80945f-5f02-4385-90af-c75abe65639b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:06.555587 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80945f-5f02-4385-90af-c75abe65639b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:06.593334 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:06.596733 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-eb80945f-5f02-4385-90af-c75abe65639b None None] MFA Rules not processed for user `c0e539c97cc04d0da396b42dc28c9919`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:06.604564 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-eb80945f-5f02-4385-90af-c75abe65639b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:06.656203 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 921/1867] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:06 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:06.656203 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915c2a2f-3c99-49f4-8aef-c93af8798bcc None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:06.696757 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 947/1868] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:06 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:06.699632 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80945f-5f02-4385-90af-c75abe65639b None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:06.701199 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-eb80945f-5f02-4385-90af-c75abe65639b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:06.722400 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:06.742125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915c2a2f-3c99-49f4-8aef-c93af8798bcc None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:06.748244 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd268bcd-5f32-4ef0-90a8-8463ae331958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:06.749091 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd268bcd-5f32-4ef0-90a8-8463ae331958 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:06.749551 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd268bcd-5f32-4ef0-90a8-8463ae331958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:06.750144 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd268bcd-5f32-4ef0-90a8-8463ae331958 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:06.751029 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd268bcd-5f32-4ef0-90a8-8463ae331958 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:06.752928 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd268bcd-5f32-4ef0-90a8-8463ae331958 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:06.766172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 922/1869] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:06.767413 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd268bcd-5f32-4ef0-90a8-8463ae331958 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:06.771639 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80945f-5f02-4385-90af-c75abe65639b None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:06.777784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b19bcec-4e1d-4756-bcf7-32c7cb31e42f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:06.778726 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19bcec-4e1d-4756-bcf7-32c7cb31e42f service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:06.779022 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19bcec-4e1d-4756-bcf7-32c7cb31e42f service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:06.779274 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd268bcd-5f32-4ef0-90a8-8463ae331958 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:06.779373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19bcec-4e1d-4756-bcf7-32c7cb31e42f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:06.786788 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a34007c-7343-4e7b-bffd-8e1693f62b23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:06.787440 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a34007c-7343-4e7b-bffd-8e1693f62b23 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:06.787726 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a34007c-7343-4e7b-bffd-8e1693f62b23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:06.788010 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a34007c-7343-4e7b-bffd-8e1693f62b23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:06.789991 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b19bcec-4e1d-4756-bcf7-32c7cb31e42f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:06.792426 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b19bcec-4e1d-4756-bcf7-32c7cb31e42f service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:06.799309 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a34007c-7343-4e7b-bffd-8e1693f62b23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:06.800678 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a34007c-7343-4e7b-bffd-8e1693f62b23 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:06.803012 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 948/1870] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:06 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:06.803909 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b19bcec-4e1d-4756-bcf7-32c7cb31e42f service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:06.816111 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b19bcec-4e1d-4756-bcf7-32c7cb31e42f service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:06.821509 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 923/1871] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:06 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:06.825468 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fae9c5-6596-4b5f-8cde-fc7c25c26445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:06.830191 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fae9c5-6596-4b5f-8cde-fc7c25c26445 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:06.830785 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fae9c5-6596-4b5f-8cde-fc7c25c26445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:06.831247 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fae9c5-6596-4b5f-8cde-fc7c25c26445 admin admin] PATH_INFO: `/v3/projects/8940150716564661aec86b7727fdfcc3/users/6ab6637397064e1b82c9ecebe3d2aa01/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:06.835347 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fae9c5-6596-4b5f-8cde-fc7c25c26445 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8940150716564661aec86b7727fdfcc3, user_id=6ab6637397064e1b82c9ecebe3d2aa01, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:06.844987 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fae9c5-6596-4b5f-8cde-fc7c25c26445 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:06.862016 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 949/1872] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:06 2022] PUT /identity/v3/projects/8940150716564661aec86b7727fdfcc3/users/6ab6637397064e1b82c9ecebe3d2aa01/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:06.872642 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6b468-bec2-40c3-9599-93831777ccc7 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:06.873135 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6b468-bec2-40c3-9599-93831777ccc7 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:06.873594 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6b468-bec2-40c3-9599-93831777ccc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:06.882505 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fae9c5-6596-4b5f-8cde-fc7c25c26445 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:06.925167 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-ace6b468-bec2-40c3-9599-93831777ccc7 None None] MFA Rules not processed for user `6ab6637397064e1b82c9ecebe3d2aa01`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:06.926903 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ace6b468-bec2-40c3-9599-93831777ccc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:06.982525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fae9c5-6596-4b5f-8cde-fc7c25c26445 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:06.987565 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e645b-16f1-462a-9a9a-1e8a460c01a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:06.988203 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e645b-16f1-462a-9a9a-1e8a460c01a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:06.988490 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e645b-16f1-462a-9a9a-1e8a460c01a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:06.988805 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e645b-16f1-462a-9a9a-1e8a460c01a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:07.033586 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 924/1873] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:06 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:07.037906 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6b468-bec2-40c3-9599-93831777ccc7 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:07.080737 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6b468-bec2-40c3-9599-93831777ccc7 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:07.080737 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401be955-d9d6-4d3a-999f-a4ae9803037c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:07.083541 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401be955-d9d6-4d3a-999f-a4ae9803037c service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:07.083964 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401be955-d9d6-4d3a-999f-a4ae9803037c service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:07.084251 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401be955-d9d6-4d3a-999f-a4ae9803037c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:07.106343 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e645b-16f1-462a-9a9a-1e8a460c01a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:07.108298 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e645b-16f1-462a-9a9a-1e8a460c01a5 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:07.121257 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 950/1874] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:07.131933 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:07.167457 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401be955-d9d6-4d3a-999f-a4ae9803037c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:07.168917 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401be955-d9d6-4d3a-999f-a4ae9803037c service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:07.178060 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 925/1875] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:07 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:07.329556 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e645b-16f1-462a-9a9a-1e8a460c01a5 service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:07.339963 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e645b-16f1-462a-9a9a-1e8a460c01a5 service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:07.355052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2206f2f-382e-4422-b059-ae83054c779e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:07.356634 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2206f2f-382e-4422-b059-ae83054c779e service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:07.356973 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2206f2f-382e-4422-b059-ae83054c779e service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:07.358455 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2206f2f-382e-4422-b059-ae83054c779e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:07.381918 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2206f2f-382e-4422-b059-ae83054c779e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:07.381918 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2206f2f-382e-4422-b059-ae83054c779e service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:07.393522 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 951/1876] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:07 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:07.875240 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401be955-d9d6-4d3a-999f-a4ae9803037c service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:07.885785 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401be955-d9d6-4d3a-999f-a4ae9803037c service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:07.889833 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf4a00-75b4-44d0-b4e7-b232dde63c36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:07.890486 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf4a00-75b4-44d0-b4e7-b232dde63c36 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:07.890767 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf4a00-75b4-44d0-b4e7-b232dde63c36 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:07.891041 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf4a00-75b4-44d0-b4e7-b232dde63c36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:07.990834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf4a00-75b4-44d0-b4e7-b232dde63c36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:07.992479 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf4a00-75b4-44d0-b4e7-b232dde63c36 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:08.027095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 926/1877] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:07 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:12.492533 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2206f2f-382e-4422-b059-ae83054c779e service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:12.604189 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:12.624578 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2206f2f-382e-4422-b059-ae83054c779e service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:12.629734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e38bc-7a1c-408a-9f52-53035efc39b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:12.630831 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e38bc-7a1c-408a-9f52-53035efc39b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:12.631277 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e38bc-7a1c-408a-9f52-53035efc39b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:12.631844 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e38bc-7a1c-408a-9f52-53035efc39b6 admin admin] PATH_INFO: `/v3/users/7912374aed7c4ea8922a8eebb9a3321e` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:12.633455 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e38bc-7a1c-408a-9f52-53035efc39b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7912374aed7c4ea8922a8eebb9a3321e)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:12.636275 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e38bc-7a1c-408a-9f52-53035efc39b6 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:12.720584 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-de9e38bc-7a1c-408a-9f52-53035efc39b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7912374aed7c4ea8922a8eebb9a3321e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:12.721725 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-de9e38bc-7a1c-408a-9f52-53035efc39b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7912374aed7c4ea8922a8eebb9a3321e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:12.723963 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-de9e38bc-7a1c-408a-9f52-53035efc39b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7912374aed7c4ea8922a8eebb9a3321e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:12.741033 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-de9e38bc-7a1c-408a-9f52-53035efc39b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7912374aed7c4ea8922a8eebb9a3321e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:12.749694 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-de9e38bc-7a1c-408a-9f52-53035efc39b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7912374aed7c4ea8922a8eebb9a3321e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:12.766395 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 952/1878] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:12 2022] DELETE /identity/v3/users/7912374aed7c4ea8922a8eebb9a3321e => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:12.771058 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8939c-4165-4495-8f28-8b4c80952732 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:12.771586 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8939c-4165-4495-8f28-8b4c80952732 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:12.771586 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8939c-4165-4495-8f28-8b4c80952732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:12.823782 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-79b8939c-4165-4495-8f28-8b4c80952732 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:12.825147 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-79b8939c-4165-4495-8f28-8b4c80952732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:12.932191 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 927/1879] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:12.953101 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e38bc-7a1c-408a-9f52-53035efc39b6 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:12.955357 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-de9e38bc-7a1c-408a-9f52-53035efc39b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:13.042841 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e38bc-7a1c-408a-9f52-53035efc39b6 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:13.047434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92652c63-9577-4413-bc38-6ce751197495 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:13.048079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92652c63-9577-4413-bc38-6ce751197495 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:13.048361 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92652c63-9577-4413-bc38-6ce751197495 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:13.048639 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92652c63-9577-4413-bc38-6ce751197495 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:13.069834 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92652c63-9577-4413-bc38-6ce751197495 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:13.071702 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92652c63-9577-4413-bc38-6ce751197495 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:13.084433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 953/1880] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:13.318282 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a9452-9b0c-4d9f-9000-fbf126b7b409 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:13.319131 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a9452-9b0c-4d9f-9000-fbf126b7b409 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:13.319131 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a9452-9b0c-4d9f-9000-fbf126b7b409 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:13.320723 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 928/1881] 10.222.0.46 () {54 vars in 921 bytes} [Wed Jul 27 04:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:22:13.335155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2dc81-9d7d-4ad7-9233-2065e4f7a308 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:13.335592 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2dc81-9d7d-4ad7-9233-2065e4f7a308 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:13.335968 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2dc81-9d7d-4ad7-9233-2065e4f7a308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:13.381644 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-7ab2dc81-9d7d-4ad7-9233-2065e4f7a308 None None] MFA Rules not processed for user `9dc9ea8f89c14a2a8e3272f750647dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:13.383837 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7ab2dc81-9d7d-4ad7-9233-2065e4f7a308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:13.499758 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:13.515340 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 954/1882] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Jul 27 04:22:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:22:13.536237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a9452-9b0c-4d9f-9000-fbf126b7b409 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:13.538220 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-122a9452-9b0c-4d9f-9000-fbf126b7b409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:13.565094 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a9452-9b0c-4d9f-9000-fbf126b7b409 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:13.572094 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca274bda-a230-4d67-b624-c50db8cdc2ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:13.572862 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca274bda-a230-4d67-b624-c50db8cdc2ef service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:13.573289 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca274bda-a230-4d67-b624-c50db8cdc2ef service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:13.573651 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca274bda-a230-4d67-b624-c50db8cdc2ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:13.586400 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca274bda-a230-4d67-b624-c50db8cdc2ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:13.589200 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca274bda-a230-4d67-b624-c50db8cdc2ef service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:13.602547 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 929/1883] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:13.801198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab2dc81-9d7d-4ad7-9233-2065e4f7a308 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:13.803194 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7ab2dc81-9d7d-4ad7-9233-2065e4f7a308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:13.824019 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab2dc81-9d7d-4ad7-9233-2065e4f7a308 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:13.828581 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab03f3e-fd8a-4a18-83b7-77900b3e99e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:13.829237 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab03f3e-fd8a-4a18-83b7-77900b3e99e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:13.829516 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab03f3e-fd8a-4a18-83b7-77900b3e99e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:13.829793 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab03f3e-fd8a-4a18-83b7-77900b3e99e8 admin admin] PATH_INFO: `/v3/projects/734a7a0ca0ed466f8c4ff18a480cdfd9` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:13.830998 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab03f3e-fd8a-4a18-83b7-77900b3e99e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=734a7a0ca0ed466f8c4ff18a480cdfd9)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:13.832942 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab03f3e-fd8a-4a18-83b7-77900b3e99e8 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:13.884102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-bab03f3e-fd8a-4a18-83b7-77900b3e99e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '734a7a0ca0ed466f8c4ff18a480cdfd9', 'request_id': 'req-bab03f3e-fd8a-4a18-83b7-77900b3e99e8'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:13.891019 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-bab03f3e-fd8a-4a18-83b7-77900b3e99e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '734a7a0ca0ed466f8c4ff18a480cdfd9', 'request_id': 'req-bab03f3e-fd8a-4a18-83b7-77900b3e99e8'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:13.896790 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-bab03f3e-fd8a-4a18-83b7-77900b3e99e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '734a7a0ca0ed466f8c4ff18a480cdfd9', 'request_id': 'req-bab03f3e-fd8a-4a18-83b7-77900b3e99e8'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:13.909916 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-bab03f3e-fd8a-4a18-83b7-77900b3e99e8 admin admin] The token cache is being invalidate because project 734a7a0ca0ed466f8c4ff18a480cdfd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:13.910176 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-bab03f3e-fd8a-4a18-83b7-77900b3e99e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:13.910806 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-bab03f3e-fd8a-4a18-83b7-77900b3e99e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:13.912602 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 955/1884] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:13 2022] DELETE /identity/v3/projects/734a7a0ca0ed466f8c4ff18a480cdfd9 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:13.917246 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca274bda-a230-4d67-b624-c50db8cdc2ef service nova] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:13.927543 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca274bda-a230-4d67-b624-c50db8cdc2ef service nova] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:13.998700 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:14.004623 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1871ca31-c1f1-4164-bad7-370f79bdae11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:14.004884 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871ca31-c1f1-4164-bad7-370f79bdae11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:14.005130 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871ca31-c1f1-4164-bad7-370f79bdae11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:14.005780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871ca31-c1f1-4164-bad7-370f79bdae11 admin admin] PATH_INFO: `/v3/users/c0e539c97cc04d0da396b42dc28c9919` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:14.008075 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1871ca31-c1f1-4164-bad7-370f79bdae11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0e539c97cc04d0da396b42dc28c9919)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:14.010893 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1871ca31-c1f1-4164-bad7-370f79bdae11 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:14.070922 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1871ca31-c1f1-4164-bad7-370f79bdae11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0e539c97cc04d0da396b42dc28c9919'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:14.071600 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1871ca31-c1f1-4164-bad7-370f79bdae11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0e539c97cc04d0da396b42dc28c9919'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:14.082069 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1871ca31-c1f1-4164-bad7-370f79bdae11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0e539c97cc04d0da396b42dc28c9919'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:14.100979 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1871ca31-c1f1-4164-bad7-370f79bdae11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0e539c97cc04d0da396b42dc28c9919'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:14.113192 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1871ca31-c1f1-4164-bad7-370f79bdae11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0e539c97cc04d0da396b42dc28c9919'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:14.124170 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 930/1885] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:13 2022] DELETE /identity/v3/users/c0e539c97cc04d0da396b42dc28c9919 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:14.421656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab03f3e-fd8a-4a18-83b7-77900b3e99e8 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:14.423174 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-bab03f3e-fd8a-4a18-83b7-77900b3e99e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:14.438097 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab03f3e-fd8a-4a18-83b7-77900b3e99e8 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:14.498605 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aeccb5-12a7-4d1f-9a1a-6bde154b7f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:14.499652 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aeccb5-12a7-4d1f-9a1a-6bde154b7f46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:14.500006 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aeccb5-12a7-4d1f-9a1a-6bde154b7f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:14.500550 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aeccb5-12a7-4d1f-9a1a-6bde154b7f46 admin admin] PATH_INFO: `/v3/projects/a16db1aafd30434fbaf6d160e889b663` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:14.502107 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aeccb5-12a7-4d1f-9a1a-6bde154b7f46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a16db1aafd30434fbaf6d160e889b663)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:14.504017 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aeccb5-12a7-4d1f-9a1a-6bde154b7f46 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:14.559231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-68aeccb5-12a7-4d1f-9a1a-6bde154b7f46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a16db1aafd30434fbaf6d160e889b663', 'request_id': 'req-68aeccb5-12a7-4d1f-9a1a-6bde154b7f46'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:14.576822 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-68aeccb5-12a7-4d1f-9a1a-6bde154b7f46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a16db1aafd30434fbaf6d160e889b663', 'request_id': 'req-68aeccb5-12a7-4d1f-9a1a-6bde154b7f46'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:14.583507 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-68aeccb5-12a7-4d1f-9a1a-6bde154b7f46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a16db1aafd30434fbaf6d160e889b663', 'request_id': 'req-68aeccb5-12a7-4d1f-9a1a-6bde154b7f46'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:14.598848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-68aeccb5-12a7-4d1f-9a1a-6bde154b7f46 admin admin] The token cache is being invalidate because project a16db1aafd30434fbaf6d160e889b663 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:14.599155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-68aeccb5-12a7-4d1f-9a1a-6bde154b7f46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:14.599844 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-68aeccb5-12a7-4d1f-9a1a-6bde154b7f46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:14.601427 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 956/1886] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:14 2022] DELETE /identity/v3/projects/a16db1aafd30434fbaf6d160e889b663 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:14.606542 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2fe50f-db89-41d5-9ae2-2c095553912c None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:14.606826 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2fe50f-db89-41d5-9ae2-2c095553912c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:14.607099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2fe50f-db89-41d5-9ae2-2c095553912c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:14.638875 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-4f2fe50f-db89-41d5-9ae2-2c095553912c None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:14.640069 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-4f2fe50f-db89-41d5-9ae2-2c095553912c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:14.699382 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 931/1887] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:14.707482 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a4c059-c587-4c5a-ad22-8ca8e95816e9 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:14.707795 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a4c059-c587-4c5a-ad22-8ca8e95816e9 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:14.708079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a4c059-c587-4c5a-ad22-8ca8e95816e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:14.727875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-08a4c059-c587-4c5a-ad22-8ca8e95816e9 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:14.730051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-08a4c059-c587-4c5a-ad22-8ca8e95816e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:14.738245 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 957/1888] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:14.742481 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2fe50f-db89-41d5-9ae2-2c095553912c None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:14.752849 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2fe50f-db89-41d5-9ae2-2c095553912c None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:14.757416 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3855fb87-cb48-48f0-b8f7-87daaf1176e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:14.758055 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3855fb87-cb48-48f0-b8f7-87daaf1176e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:14.758335 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3855fb87-cb48-48f0-b8f7-87daaf1176e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:14.758604 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3855fb87-cb48-48f0-b8f7-87daaf1176e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:14.759297 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3855fb87-cb48-48f0-b8f7-87daaf1176e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:22:14.759658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3855fb87-cb48-48f0-b8f7-87daaf1176e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:14.761217 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3855fb87-cb48-48f0-b8f7-87daaf1176e0 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:14.775188 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 932/1889] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:22:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:22:14.779552 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a4c059-c587-4c5a-ad22-8ca8e95816e9 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:14.789062 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a4c059-c587-4c5a-ad22-8ca8e95816e9 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:14.794469 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa831ad4-a341-4089-868a-f11dc93e1c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:14.795091 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa831ad4-a341-4089-868a-f11dc93e1c38 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:14.795372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa831ad4-a341-4089-868a-f11dc93e1c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:14.795765 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa831ad4-a341-4089-868a-f11dc93e1c38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:14.796303 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa831ad4-a341-4089-868a-f11dc93e1c38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:14.798216 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa831ad4-a341-4089-868a-f11dc93e1c38 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:14.802905 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:14.829443 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 958/1890] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:14 2022] POST /identity/v3/projects => generated 369 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:14.833465 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3855fb87-cb48-48f0-b8f7-87daaf1176e0 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:14.843339 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3855fb87-cb48-48f0-b8f7-87daaf1176e0 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:14.898061 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6cb61e-aaff-4d8f-a2cf-686be12335b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:14.898619 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6cb61e-aaff-4d8f-a2cf-686be12335b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:14.898907 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6cb61e-aaff-4d8f-a2cf-686be12335b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:14.899173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6cb61e-aaff-4d8f-a2cf-686be12335b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:14.899829 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6cb61e-aaff-4d8f-a2cf-686be12335b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:14.901796 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6cb61e-aaff-4d8f-a2cf-686be12335b8 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:14.909488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:14.924474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 933/1891] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:14 2022] POST /identity/v3/users => generated 285 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:14.928750 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa831ad4-a341-4089-868a-f11dc93e1c38 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:14.940779 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa831ad4-a341-4089-868a-f11dc93e1c38 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:14.949599 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a93ebde-b336-41fe-998a-5d863e8bd563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:14.950352 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93ebde-b336-41fe-998a-5d863e8bd563 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:14.950642 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93ebde-b336-41fe-998a-5d863e8bd563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:14.950965 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93ebde-b336-41fe-998a-5d863e8bd563 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:14.951762 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a93ebde-b336-41fe-998a-5d863e8bd563 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:14.953628 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a93ebde-b336-41fe-998a-5d863e8bd563 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:14.966968 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 959/1892] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:14.970975 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6cb61e-aaff-4d8f-a2cf-686be12335b8 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:14.981944 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6cb61e-aaff-4d8f-a2cf-686be12335b8 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:14.986578 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb01ca-67e7-4207-a56e-3092dd03dad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:14.987193 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb01ca-67e7-4207-a56e-3092dd03dad9 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:14.987480 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb01ca-67e7-4207-a56e-3092dd03dad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:14.987760 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb01ca-67e7-4207-a56e-3092dd03dad9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:14.988495 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb01ca-67e7-4207-a56e-3092dd03dad9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:14.990181 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb01ca-67e7-4207-a56e-3092dd03dad9 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:15.003206 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 934/1893] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 04:22:15.009132 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a93ebde-b336-41fe-998a-5d863e8bd563 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:15.019753 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a93ebde-b336-41fe-998a-5d863e8bd563 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:15.024646 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb6b4f4-1561-4a1e-8672-0b680a78bc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:15.025961 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6b4f4-1561-4a1e-8672-0b680a78bc75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:15.026407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6b4f4-1561-4a1e-8672-0b680a78bc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:15.026792 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6b4f4-1561-4a1e-8672-0b680a78bc75 admin admin] PATH_INFO: `/v3/projects/a6e0bd3ac291421ba0000d6055f4e916/users/4237539fffea4a9a878345a12850e2cf/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:15.046605 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb6b4f4-1561-4a1e-8672-0b680a78bc75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6e0bd3ac291421ba0000d6055f4e916, user_id=4237539fffea4a9a878345a12850e2cf, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:15.049438 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb6b4f4-1561-4a1e-8672-0b680a78bc75 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:15.074322 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 960/1894] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:15 2022] PUT /identity/v3/projects/a6e0bd3ac291421ba0000d6055f4e916/users/4237539fffea4a9a878345a12850e2cf/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:15.082009 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129d447-bf68-415d-82ff-2d3a126f4af6 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:15.082301 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129d447-bf68-415d-82ff-2d3a126f4af6 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:15.082578 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129d447-bf68-415d-82ff-2d3a126f4af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:15.118330 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-6129d447-bf68-415d-82ff-2d3a126f4af6 None None] MFA Rules not processed for user `4237539fffea4a9a878345a12850e2cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:15.120180 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-6129d447-bf68-415d-82ff-2d3a126f4af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:15.200141 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 935/1895] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:15 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:15.204670 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb6b4f4-1561-4a1e-8672-0b680a78bc75 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:15.288229 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb6b4f4-1561-4a1e-8672-0b680a78bc75 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:15.297495 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5395d-0ba5-4655-9722-a55e723a860c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:15.298557 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5395d-0ba5-4655-9722-a55e723a860c admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:15.299044 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5395d-0ba5-4655-9722-a55e723a860c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:15.299567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5395d-0ba5-4655-9722-a55e723a860c admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:15.301940 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa5395d-0ba5-4655-9722-a55e723a860c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:15.304499 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa5395d-0ba5-4655-9722-a55e723a860c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:15.342931 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 961/1896] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:15 2022] POST /identity/v3/projects => generated 369 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:15.346331 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6129d447-bf68-415d-82ff-2d3a126f4af6 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:15.366741 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6129d447-bf68-415d-82ff-2d3a126f4af6 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:15.445820 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407de75-4fc0-4b2e-b1ec-c27f46f26d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:15.446558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407de75-4fc0-4b2e-b1ec-c27f46f26d6a admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:15.447449 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407de75-4fc0-4b2e-b1ec-c27f46f26d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:15.447950 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407de75-4fc0-4b2e-b1ec-c27f46f26d6a admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:15.449004 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2407de75-4fc0-4b2e-b1ec-c27f46f26d6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:15.451206 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2407de75-4fc0-4b2e-b1ec-c27f46f26d6a admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:15.498643 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 936/1897] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:15 2022] POST /identity/v3/users => generated 291 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:22:15.502019 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5395d-0ba5-4655-9722-a55e723a860c admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:15.521643 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5395d-0ba5-4655-9722-a55e723a860c admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:15.530804 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fafb31-8676-4007-aac9-b123fefbf0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:15.532559 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fafb31-8676-4007-aac9-b123fefbf0a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:15.533197 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fafb31-8676-4007-aac9-b123fefbf0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:15.533666 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fafb31-8676-4007-aac9-b123fefbf0a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:15.535077 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fafb31-8676-4007-aac9-b123fefbf0a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:15.538565 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fafb31-8676-4007-aac9-b123fefbf0a8 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:15.558866 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 962/1898] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:15.559485 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407de75-4fc0-4b2e-b1ec-c27f46f26d6a admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:15.579288 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407de75-4fc0-4b2e-b1ec-c27f46f26d6a admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:15.586905 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ae528-a91d-4daa-8c8c-b0d6dc5507c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:15.588658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ae528-a91d-4daa-8c8c-b0d6dc5507c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:15.589051 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ae528-a91d-4daa-8c8c-b0d6dc5507c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:15.589431 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ae528-a91d-4daa-8c8c-b0d6dc5507c0 admin admin] PATH_INFO: `/v3/projects/c7486616f13749079d7173b9af0c0cc6/users/a67ef3f283954e6ba998115593e65b3a/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:15.611395 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ae528-a91d-4daa-8c8c-b0d6dc5507c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7486616f13749079d7173b9af0c0cc6, user_id=a67ef3f283954e6ba998115593e65b3a, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:15.615256 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ae528-a91d-4daa-8c8c-b0d6dc5507c0 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:15.651797 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 937/1899] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:15 2022] PUT /identity/v3/projects/c7486616f13749079d7173b9af0c0cc6/users/a67ef3f283954e6ba998115593e65b3a/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:22:15.655736 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fafb31-8676-4007-aac9-b123fefbf0a8 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:15.690180 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fafb31-8676-4007-aac9-b123fefbf0a8 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:15.820439 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f7d02b-f0ac-4833-b770-2ba17b9897f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:15.822626 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7d02b-f0ac-4833-b770-2ba17b9897f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:15.822965 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7d02b-f0ac-4833-b770-2ba17b9897f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:15.823255 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7d02b-f0ac-4833-b770-2ba17b9897f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:15.823973 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f7d02b-f0ac-4833-b770-2ba17b9897f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:15.829216 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f7d02b-f0ac-4833-b770-2ba17b9897f1 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:15.860458 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 963/1900] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:15.861804 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ae528-a91d-4daa-8c8c-b0d6dc5507c0 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:15.877754 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ae528-a91d-4daa-8c8c-b0d6dc5507c0 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:15.888277 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38f63c-eeec-4548-835f-95529c8263e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:15.889152 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38f63c-eeec-4548-835f-95529c8263e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:15.889451 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38f63c-eeec-4548-835f-95529c8263e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:15.889828 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38f63c-eeec-4548-835f-95529c8263e1 admin admin] PATH_INFO: `/v3/projects/c7486616f13749079d7173b9af0c0cc6/users/a67ef3f283954e6ba998115593e65b3a/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:15.902449 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c38f63c-eeec-4548-835f-95529c8263e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7486616f13749079d7173b9af0c0cc6, user_id=a67ef3f283954e6ba998115593e65b3a, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:15.905301 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c38f63c-eeec-4548-835f-95529c8263e1 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:15.926158 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 938/1901] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:15 2022] PUT /identity/v3/projects/c7486616f13749079d7173b9af0c0cc6/users/a67ef3f283954e6ba998115593e65b3a/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:15.938034 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab478b-84bc-46ca-b04a-1ce39bf52374 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:15.938675 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab478b-84bc-46ca-b04a-1ce39bf52374 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:15.939135 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab478b-84bc-46ca-b04a-1ce39bf52374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:15.983577 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-7cab478b-84bc-46ca-b04a-1ce39bf52374 None None] MFA Rules not processed for user `a67ef3f283954e6ba998115593e65b3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:15.983773 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7cab478b-84bc-46ca-b04a-1ce39bf52374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:15.992862 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:16.141270 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38f63c-eeec-4548-835f-95529c8263e1 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:16.145242 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-1c38f63c-eeec-4548-835f-95529c8263e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:16.148264 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 964/1902] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:15 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:16.183760 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:16.240422 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38f63c-eeec-4548-835f-95529c8263e1 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:16.250452 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56292277-8e43-437c-baef-c778b6a88b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:16.251459 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56292277-8e43-437c-baef-c778b6a88b9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:16.252832 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56292277-8e43-437c-baef-c778b6a88b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:16.253343 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56292277-8e43-437c-baef-c778b6a88b9a admin admin] PATH_INFO: `/v3/users/db44c99bb91a4da1ab75f9e7194be739` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:16.255362 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56292277-8e43-437c-baef-c778b6a88b9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=db44c99bb91a4da1ab75f9e7194be739)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:16.259350 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56292277-8e43-437c-baef-c778b6a88b9a admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:16.288658 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab478b-84bc-46ca-b04a-1ce39bf52374 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:16.291988 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7cab478b-84bc-46ca-b04a-1ce39bf52374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:16.337533 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-56292277-8e43-437c-baef-c778b6a88b9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db44c99bb91a4da1ab75f9e7194be739'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:16.341225 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-56292277-8e43-437c-baef-c778b6a88b9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db44c99bb91a4da1ab75f9e7194be739'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:16.351764 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-56292277-8e43-437c-baef-c778b6a88b9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db44c99bb91a4da1ab75f9e7194be739'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:16.382640 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab478b-84bc-46ca-b04a-1ce39bf52374 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:16.382962 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-56292277-8e43-437c-baef-c778b6a88b9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db44c99bb91a4da1ab75f9e7194be739'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:16.387503 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b4fef6-a87e-4f1e-88c1-db87e9666897 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:16.388265 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4fef6-a87e-4f1e-88c1-db87e9666897 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:16.388647 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4fef6-a87e-4f1e-88c1-db87e9666897 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:16.389027 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4fef6-a87e-4f1e-88c1-db87e9666897 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:16.392540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-56292277-8e43-437c-baef-c778b6a88b9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db44c99bb91a4da1ab75f9e7194be739'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:16.399430 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b4fef6-a87e-4f1e-88c1-db87e9666897 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:16.404308 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 939/1903] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:16 2022] DELETE /identity/v3/users/db44c99bb91a4da1ab75f9e7194be739 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:16.404494 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b4fef6-a87e-4f1e-88c1-db87e9666897 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:16.405881 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5941d-9ae7-4e43-a1b5-5a90993d4432 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:16.406325 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5941d-9ae7-4e43-a1b5-5a90993d4432 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:16.406531 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5941d-9ae7-4e43-a1b5-5a90993d4432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:16.414196 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 965/1904] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:16 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:16.428037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-79c5941d-9ae7-4e43-a1b5-5a90993d4432 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:16.429262 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-79c5941d-9ae7-4e43-a1b5-5a90993d4432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:16.533877 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 940/1905] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:16.544445 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b4fef6-a87e-4f1e-88c1-db87e9666897 service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:16.546324 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-08b4fef6-a87e-4f1e-88c1-db87e9666897 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:16.631620 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b4fef6-a87e-4f1e-88c1-db87e9666897 service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:16.722142 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ab97f-e890-4e22-b802-6e7f53b2b4d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:16.723373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ab97f-e890-4e22-b802-6e7f53b2b4d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:16.723674 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ab97f-e890-4e22-b802-6e7f53b2b4d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:16.723949 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ab97f-e890-4e22-b802-6e7f53b2b4d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:16.742569 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ab97f-e890-4e22-b802-6e7f53b2b4d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:16.742569 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ab97f-e890-4e22-b802-6e7f53b2b4d3 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:16.750398 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 966/1906] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:17.069227 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5941d-9ae7-4e43-a1b5-5a90993d4432 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:17.071024 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-79c5941d-9ae7-4e43-a1b5-5a90993d4432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:17.085077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5941d-9ae7-4e43-a1b5-5a90993d4432 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:17.093374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f508c2-eaf8-489f-8475-6b88a72dd693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:17.094274 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f508c2-eaf8-489f-8475-6b88a72dd693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:17.094561 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f508c2-eaf8-489f-8475-6b88a72dd693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:17.094846 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f508c2-eaf8-489f-8475-6b88a72dd693 admin admin] PATH_INFO: `/v3/projects/14f8f6245b4244998d4a30c5717a1480` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:17.097273 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f508c2-eaf8-489f-8475-6b88a72dd693 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14f8f6245b4244998d4a30c5717a1480)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:17.099390 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f508c2-eaf8-489f-8475-6b88a72dd693 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:17.141327 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:17.153540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c4f508c2-eaf8-489f-8475-6b88a72dd693 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14f8f6245b4244998d4a30c5717a1480', 'request_id': 'req-c4f508c2-eaf8-489f-8475-6b88a72dd693'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:17.161194 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c4f508c2-eaf8-489f-8475-6b88a72dd693 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14f8f6245b4244998d4a30c5717a1480', 'request_id': 'req-c4f508c2-eaf8-489f-8475-6b88a72dd693'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:17.167640 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c4f508c2-eaf8-489f-8475-6b88a72dd693 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14f8f6245b4244998d4a30c5717a1480', 'request_id': 'req-c4f508c2-eaf8-489f-8475-6b88a72dd693'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:17.183406 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c4f508c2-eaf8-489f-8475-6b88a72dd693 admin admin] The token cache is being invalidate because project 14f8f6245b4244998d4a30c5717a1480 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:17.183882 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c4f508c2-eaf8-489f-8475-6b88a72dd693 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:17.184625 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c4f508c2-eaf8-489f-8475-6b88a72dd693 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:17.186519 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 941/1907] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:17 2022] DELETE /identity/v3/projects/14f8f6245b4244998d4a30c5717a1480 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:17.190663 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ab97f-e890-4e22-b802-6e7f53b2b4d3 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:17.202188 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ab97f-e890-4e22-b802-6e7f53b2b4d3 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:17.211780 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:17.288091 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a7009-3e9c-4957-a8ad-65656b7d9809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:17.288873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a7009-3e9c-4957-a8ad-65656b7d9809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:17.289169 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a7009-3e9c-4957-a8ad-65656b7d9809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:17.289483 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a7009-3e9c-4957-a8ad-65656b7d9809 admin admin] PATH_INFO: `/v3/users/226618f2857c4f44ba7228ebf1f5a2a6` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:17.291919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a7009-3e9c-4957-a8ad-65656b7d9809 admin admin] RBAC: Authorizing `identity:delete_user(user_id=226618f2857c4f44ba7228ebf1f5a2a6)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:17.294242 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a7009-3e9c-4957-a8ad-65656b7d9809 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:17.351784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-301a7009-3e9c-4957-a8ad-65656b7d9809 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '226618f2857c4f44ba7228ebf1f5a2a6'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:17.352475 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-301a7009-3e9c-4957-a8ad-65656b7d9809 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '226618f2857c4f44ba7228ebf1f5a2a6'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:17.353240 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-301a7009-3e9c-4957-a8ad-65656b7d9809 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '226618f2857c4f44ba7228ebf1f5a2a6'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:17.367138 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-301a7009-3e9c-4957-a8ad-65656b7d9809 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '226618f2857c4f44ba7228ebf1f5a2a6'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:17.374677 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-301a7009-3e9c-4957-a8ad-65656b7d9809 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '226618f2857c4f44ba7228ebf1f5a2a6'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:17.388884 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 967/1908] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:17 2022] DELETE /identity/v3/users/226618f2857c4f44ba7228ebf1f5a2a6 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:17.934285 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f508c2-eaf8-489f-8475-6b88a72dd693 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:17.934894 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c4f508c2-eaf8-489f-8475-6b88a72dd693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:17.966458 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f508c2-eaf8-489f-8475-6b88a72dd693 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:18.030844 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2286fd2f-7059-40e5-b4df-e85289067be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:18.031514 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286fd2f-7059-40e5-b4df-e85289067be1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:18.031794 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286fd2f-7059-40e5-b4df-e85289067be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:18.032162 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286fd2f-7059-40e5-b4df-e85289067be1 admin admin] PATH_INFO: `/v3/projects/4598ce65aafd48499d3f790e58d531f9` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:18.033224 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2286fd2f-7059-40e5-b4df-e85289067be1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4598ce65aafd48499d3f790e58d531f9)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:18.035149 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2286fd2f-7059-40e5-b4df-e85289067be1 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:18.093911 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2286fd2f-7059-40e5-b4df-e85289067be1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4598ce65aafd48499d3f790e58d531f9', 'request_id': 'req-2286fd2f-7059-40e5-b4df-e85289067be1'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:18.101156 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2286fd2f-7059-40e5-b4df-e85289067be1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4598ce65aafd48499d3f790e58d531f9', 'request_id': 'req-2286fd2f-7059-40e5-b4df-e85289067be1'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:18.107324 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2286fd2f-7059-40e5-b4df-e85289067be1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4598ce65aafd48499d3f790e58d531f9', 'request_id': 'req-2286fd2f-7059-40e5-b4df-e85289067be1'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:18.121616 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2286fd2f-7059-40e5-b4df-e85289067be1 admin admin] The token cache is being invalidate because project 4598ce65aafd48499d3f790e58d531f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:18.121903 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2286fd2f-7059-40e5-b4df-e85289067be1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:18.122594 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2286fd2f-7059-40e5-b4df-e85289067be1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:18.124601 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 942/1909] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:17 2022] DELETE /identity/v3/projects/4598ce65aafd48499d3f790e58d531f9 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:22:18.130116 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2caa4-27bf-40a3-ae63-4e5dcfb88582 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:18.130608 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2caa4-27bf-40a3-ae63-4e5dcfb88582 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:18.130708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2caa4-27bf-40a3-ae63-4e5dcfb88582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:18.150868 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-e5d2caa4-27bf-40a3-ae63-4e5dcfb88582 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:18.152135 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e5d2caa4-27bf-40a3-ae63-4e5dcfb88582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:18.217230 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 968/1910] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:18.225695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42459f-681a-4266-bb75-f208f9255223 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:18.226161 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42459f-681a-4266-bb75-f208f9255223 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:18.226463 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42459f-681a-4266-bb75-f208f9255223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:18.249326 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-df42459f-681a-4266-bb75-f208f9255223 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:18.250577 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-df42459f-681a-4266-bb75-f208f9255223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:18.256115 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d2caa4-27bf-40a3-ae63-4e5dcfb88582 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:18.258701 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e5d2caa4-27bf-40a3-ae63-4e5dcfb88582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:18.259968 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 943/1911] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:18.264075 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42459f-681a-4266-bb75-f208f9255223 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:18.273485 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42459f-681a-4266-bb75-f208f9255223 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:18.273901 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d2caa4-27bf-40a3-ae63-4e5dcfb88582 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:18.278527 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5fdfb3-698f-46e7-aaab-72b4cb8bb726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:18.278817 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f2c8a-798c-4f6d-a880-9c6ca125fcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:18.279134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fdfb3-698f-46e7-aaab-72b4cb8bb726 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:18.279441 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fdfb3-698f-46e7-aaab-72b4cb8bb726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:18.279714 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fdfb3-698f-46e7-aaab-72b4cb8bb726 admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:18.279804 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f2c8a-798c-4f6d-a880-9c6ca125fcbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:18.280179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f2c8a-798c-4f6d-a880-9c6ca125fcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:18.280410 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5fdfb3-698f-46e7-aaab-72b4cb8bb726 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:22:18.280633 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f2c8a-798c-4f6d-a880-9c6ca125fcbb admin admin] PATH_INFO: `/v3/users/4237539fffea4a9a878345a12850e2cf` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:18.280735 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5fdfb3-698f-46e7-aaab-72b4cb8bb726 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:18.281995 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f2c8a-798c-4f6d-a880-9c6ca125fcbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4237539fffea4a9a878345a12850e2cf)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:18.282347 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5fdfb3-698f-46e7-aaab-72b4cb8bb726 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:18.284065 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f2c8a-798c-4f6d-a880-9c6ca125fcbb admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:18.294300 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 944/1912] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:22:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:22:18.298529 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5fdfb3-698f-46e7-aaab-72b4cb8bb726 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:18.308564 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5fdfb3-698f-46e7-aaab-72b4cb8bb726 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:18.309708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:18.313845 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c038b532-6a3f-48fb-9860-ea43af745a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:18.314483 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038b532-6a3f-48fb-9860-ea43af745a46 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:18.315168 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038b532-6a3f-48fb-9860-ea43af745a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:18.315168 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038b532-6a3f-48fb-9860-ea43af745a46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:18.315690 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c038b532-6a3f-48fb-9860-ea43af745a46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:18.317570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c038b532-6a3f-48fb-9860-ea43af745a46 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:18.336273 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ac9f2c8a-798c-4f6d-a880-9c6ca125fcbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4237539fffea4a9a878345a12850e2cf'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:18.336969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ac9f2c8a-798c-4f6d-a880-9c6ca125fcbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4237539fffea4a9a878345a12850e2cf'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:18.337822 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ac9f2c8a-798c-4f6d-a880-9c6ca125fcbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4237539fffea4a9a878345a12850e2cf'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:18.350610 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 945/1913] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:18 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:22:18.352427 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c038b532-6a3f-48fb-9860-ea43af745a46 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:18.352841 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ac9f2c8a-798c-4f6d-a880-9c6ca125fcbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4237539fffea4a9a878345a12850e2cf'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:18.354162 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c038b532-6a3f-48fb-9860-ea43af745a46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:18.360572 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ac9f2c8a-798c-4f6d-a880-9c6ca125fcbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4237539fffea4a9a878345a12850e2cf'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:18.368132 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:18.382017 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 969/1914] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:18 2022] DELETE /identity/v3/users/4237539fffea4a9a878345a12850e2cf => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:18.383352 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25522cbd-27c6-441f-9fcf-b377ad1971cc None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:18.384359 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25522cbd-27c6-441f-9fcf-b377ad1971cc None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:18.384359 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25522cbd-27c6-441f-9fcf-b377ad1971cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:18.406194 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-25522cbd-27c6-441f-9fcf-b377ad1971cc None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:18.407544 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-25522cbd-27c6-441f-9fcf-b377ad1971cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:18.436332 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c038b532-6a3f-48fb-9860-ea43af745a46 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:18.492812 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 970/1915] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:22:18.504872 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25522cbd-27c6-441f-9fcf-b377ad1971cc None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:18.506442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-25522cbd-27c6-441f-9fcf-b377ad1971cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:18.507495 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c22c63-a749-4e8c-bb2b-d91941834aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:18.508195 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c22c63-a749-4e8c-bb2b-d91941834aaa admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:18.508606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c22c63-a749-4e8c-bb2b-d91941834aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:18.509024 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c22c63-a749-4e8c-bb2b-d91941834aaa admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:18.509836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c22c63-a749-4e8c-bb2b-d91941834aaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:18.512077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c22c63-a749-4e8c-bb2b-d91941834aaa admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:18.535542 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 946/1916] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:18 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:18.539626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c22c63-a749-4e8c-bb2b-d91941834aaa admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:18.551115 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c22c63-a749-4e8c-bb2b-d91941834aaa admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:18.565947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7dd60e-3e76-4790-b248-8ef2f622da30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:18.566692 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7dd60e-3e76-4790-b248-8ef2f622da30 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:18.567053 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7dd60e-3e76-4790-b248-8ef2f622da30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:18.567405 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7dd60e-3e76-4790-b248-8ef2f622da30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:18.568142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7dd60e-3e76-4790-b248-8ef2f622da30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:18.570259 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7dd60e-3e76-4790-b248-8ef2f622da30 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:18.584208 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 947/1917] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 04:22:18.584416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25522cbd-27c6-441f-9fcf-b377ad1971cc None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:18.585197 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7dd60e-3e76-4790-b248-8ef2f622da30 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:18.587771 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016ca6a-a90f-445f-b4e8-eb619a15ecff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:18.588487 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016ca6a-a90f-445f-b4e8-eb619a15ecff service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:18.588913 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016ca6a-a90f-445f-b4e8-eb619a15ecff service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:18.589268 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016ca6a-a90f-445f-b4e8-eb619a15ecff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:18.596285 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7dd60e-3e76-4790-b248-8ef2f622da30 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:18.600178 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3016ca6a-a90f-445f-b4e8-eb619a15ecff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:18.601413 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006bbf56-0b40-4081-a8a7-6b9774a58064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:18.601620 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3016ca6a-a90f-445f-b4e8-eb619a15ecff service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:18.602413 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bbf56-0b40-4081-a8a7-6b9774a58064 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:18.602878 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bbf56-0b40-4081-a8a7-6b9774a58064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:18.603874 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bbf56-0b40-4081-a8a7-6b9774a58064 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:18.604730 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006bbf56-0b40-4081-a8a7-6b9774a58064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:18.606736 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006bbf56-0b40-4081-a8a7-6b9774a58064 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:18.615073 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 971/1918] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 04:22:18.625077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 948/1919] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:18.626309 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016ca6a-a90f-445f-b4e8-eb619a15ecff service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:18.636511 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016ca6a-a90f-445f-b4e8-eb619a15ecff service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:18.641346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d199ba-c85f-476e-be46-3665550eae34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:18.642059 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d199ba-c85f-476e-be46-3665550eae34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:18.642377 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d199ba-c85f-476e-be46-3665550eae34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:18.642659 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d199ba-c85f-476e-be46-3665550eae34 admin admin] PATH_INFO: `/v3/projects/cc6cf109e9154af5a7e85431ee5e5c31/users/2291aad176c345d6a6fd116af02c8156/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:18.660547 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d199ba-c85f-476e-be46-3665550eae34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc6cf109e9154af5a7e85431ee5e5c31, user_id=2291aad176c345d6a6fd116af02c8156, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:18.663757 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d199ba-c85f-476e-be46-3665550eae34 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:18.678975 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 972/1920] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:18 2022] PUT /identity/v3/projects/cc6cf109e9154af5a7e85431ee5e5c31/users/2291aad176c345d6a6fd116af02c8156/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:18.690045 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8159b6f-ca98-4595-94ca-22c5f6a3f7db None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:18.690343 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8159b6f-ca98-4595-94ca-22c5f6a3f7db None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:18.690627 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8159b6f-ca98-4595-94ca-22c5f6a3f7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:18.735234 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-f8159b6f-ca98-4595-94ca-22c5f6a3f7db None None] MFA Rules not processed for user `2291aad176c345d6a6fd116af02c8156`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:18.736470 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f8159b6f-ca98-4595-94ca-22c5f6a3f7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:18.813194 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 949/1921] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:18.824925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d199ba-c85f-476e-be46-3665550eae34 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:18.826668 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e4d199ba-c85f-476e-be46-3665550eae34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:18.848393 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:18.892375 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d199ba-c85f-476e-be46-3665550eae34 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:18.896151 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2e635-f68d-4d66-9457-5b8c4c1e7111 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:18.896783 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2e635-f68d-4d66-9457-5b8c4c1e7111 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:18.897068 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2e635-f68d-4d66-9457-5b8c4c1e7111 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:18.897369 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2e635-f68d-4d66-9457-5b8c4c1e7111 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:18.906943 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8159b6f-ca98-4595-94ca-22c5f6a3f7db None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:18.908669 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f8159b6f-ca98-4595-94ca-22c5f6a3f7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:18.910085 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed2e635-f68d-4d66-9457-5b8c4c1e7111 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:18.911703 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed2e635-f68d-4d66-9457-5b8c4c1e7111 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:18.922799 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 973/1922] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:18.941151 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:19.003132 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8159b6f-ca98-4595-94ca-22c5f6a3f7db None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:19.008297 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbd799b-19b0-4091-b201-4b7a3bcca09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:19.008958 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd799b-19b0-4091-b201-4b7a3bcca09f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:19.009300 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd799b-19b0-4091-b201-4b7a3bcca09f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:19.009655 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd799b-19b0-4091-b201-4b7a3bcca09f admin admin] PATH_INFO: `/v3/projects/a6e0bd3ac291421ba0000d6055f4e916` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:19.010919 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbd799b-19b0-4091-b201-4b7a3bcca09f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6e0bd3ac291421ba0000d6055f4e916)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:19.012918 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbd799b-19b0-4091-b201-4b7a3bcca09f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:19.069131 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0fbd799b-19b0-4091-b201-4b7a3bcca09f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6e0bd3ac291421ba0000d6055f4e916', 'request_id': 'req-0fbd799b-19b0-4091-b201-4b7a3bcca09f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:19.076987 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0fbd799b-19b0-4091-b201-4b7a3bcca09f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6e0bd3ac291421ba0000d6055f4e916', 'request_id': 'req-0fbd799b-19b0-4091-b201-4b7a3bcca09f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:19.083272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0fbd799b-19b0-4091-b201-4b7a3bcca09f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6e0bd3ac291421ba0000d6055f4e916', 'request_id': 'req-0fbd799b-19b0-4091-b201-4b7a3bcca09f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:19.097867 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0fbd799b-19b0-4091-b201-4b7a3bcca09f admin admin] The token cache is being invalidate because project a6e0bd3ac291421ba0000d6055f4e916 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:19.098455 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0fbd799b-19b0-4091-b201-4b7a3bcca09f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:19.099275 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0fbd799b-19b0-4091-b201-4b7a3bcca09f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:19.101035 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 950/1923] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:18 2022] DELETE /identity/v3/projects/a6e0bd3ac291421ba0000d6055f4e916 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:19.105211 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2e635-f68d-4d66-9457-5b8c4c1e7111 service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:19.118081 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2e635-f68d-4d66-9457-5b8c4c1e7111 service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:19.206251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7fda85-86f9-4ebc-b383-da824a0cc750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:19.206251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fda85-86f9-4ebc-b383-da824a0cc750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:19.206251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fda85-86f9-4ebc-b383-da824a0cc750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:19.206251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fda85-86f9-4ebc-b383-da824a0cc750 admin admin] PATH_INFO: `/v3/users/a67ef3f283954e6ba998115593e65b3a` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:19.206980 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7fda85-86f9-4ebc-b383-da824a0cc750 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a67ef3f283954e6ba998115593e65b3a)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:19.206980 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7fda85-86f9-4ebc-b383-da824a0cc750 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:19.268720 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9c7fda85-86f9-4ebc-b383-da824a0cc750 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a67ef3f283954e6ba998115593e65b3a'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:19.269415 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9c7fda85-86f9-4ebc-b383-da824a0cc750 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a67ef3f283954e6ba998115593e65b3a'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:19.270200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9c7fda85-86f9-4ebc-b383-da824a0cc750 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a67ef3f283954e6ba998115593e65b3a'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:19.295375 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9c7fda85-86f9-4ebc-b383-da824a0cc750 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a67ef3f283954e6ba998115593e65b3a'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:19.295375 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9c7fda85-86f9-4ebc-b383-da824a0cc750 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a67ef3f283954e6ba998115593e65b3a'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:19.310491 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 974/1924] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:19 2022] DELETE /identity/v3/users/a67ef3f283954e6ba998115593e65b3a => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:19.665617 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbd799b-19b0-4091-b201-4b7a3bcca09f admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:19.668196 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0fbd799b-19b0-4091-b201-4b7a3bcca09f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:19.689419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbd799b-19b0-4091-b201-4b7a3bcca09f admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:19.797508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7eada9-f847-4b1b-a2cf-9924bcc7d4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:19.798340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7eada9-f847-4b1b-a2cf-9924bcc7d4d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:19.798692 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7eada9-f847-4b1b-a2cf-9924bcc7d4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:19.799042 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7eada9-f847-4b1b-a2cf-9924bcc7d4d9 admin admin] PATH_INFO: `/v3/projects/c7486616f13749079d7173b9af0c0cc6` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:19.800971 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7eada9-f847-4b1b-a2cf-9924bcc7d4d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7486616f13749079d7173b9af0c0cc6)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:19.803278 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7eada9-f847-4b1b-a2cf-9924bcc7d4d9 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:19.888604 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:19.890192 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-7e7eada9-f847-4b1b-a2cf-9924bcc7d4d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7486616f13749079d7173b9af0c0cc6', 'request_id': 'req-7e7eada9-f847-4b1b-a2cf-9924bcc7d4d9'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:19.899520 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-7e7eada9-f847-4b1b-a2cf-9924bcc7d4d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7486616f13749079d7173b9af0c0cc6', 'request_id': 'req-7e7eada9-f847-4b1b-a2cf-9924bcc7d4d9'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:19.909964 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-7e7eada9-f847-4b1b-a2cf-9924bcc7d4d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7486616f13749079d7173b9af0c0cc6', 'request_id': 'req-7e7eada9-f847-4b1b-a2cf-9924bcc7d4d9'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:19.928788 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-7e7eada9-f847-4b1b-a2cf-9924bcc7d4d9 admin admin] The token cache is being invalidate because project c7486616f13749079d7173b9af0c0cc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:19.929177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-7e7eada9-f847-4b1b-a2cf-9924bcc7d4d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:19.929963 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-7e7eada9-f847-4b1b-a2cf-9924bcc7d4d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:19.934008 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 951/1925] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:19 2022] DELETE /identity/v3/projects/c7486616f13749079d7173b9af0c0cc6 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:19.936845 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4b699-da43-4570-b84a-cdc9fedc3675 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:19.937134 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4b699-da43-4570-b84a-cdc9fedc3675 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:19.937413 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4b699-da43-4570-b84a-cdc9fedc3675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:19.957678 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-3ff4b699-da43-4570-b84a-cdc9fedc3675 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:19.958938 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-3ff4b699-da43-4570-b84a-cdc9fedc3675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:19.994297 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:20.024413 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 975/1926] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:20.033358 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf08be6-aea3-4edb-8ed8-345f93b947ef None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:20.033653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf08be6-aea3-4edb-8ed8-345f93b947ef None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:20.033999 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf08be6-aea3-4edb-8ed8-345f93b947ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:20.055966 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-6bf08be6-aea3-4edb-8ed8-345f93b947ef None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:20.057561 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-6bf08be6-aea3-4edb-8ed8-345f93b947ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:20.065764 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 952/1927] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:20.070411 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff4b699-da43-4570-b84a-cdc9fedc3675 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:20.080636 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff4b699-da43-4570-b84a-cdc9fedc3675 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:20.085617 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b463f1-a370-4c3a-b534-8cb41d6c8e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:20.086333 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b463f1-a370-4c3a-b534-8cb41d6c8e25 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:20.086613 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b463f1-a370-4c3a-b534-8cb41d6c8e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:20.086886 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b463f1-a370-4c3a-b534-8cb41d6c8e25 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:20.088225 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b463f1-a370-4c3a-b534-8cb41d6c8e25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:22:20.088535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b463f1-a370-4c3a-b534-8cb41d6c8e25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:20.094170 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b463f1-a370-4c3a-b534-8cb41d6c8e25 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:20.102496 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 976/1928] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:22:20.107031 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf08be6-aea3-4edb-8ed8-345f93b947ef None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:20.115965 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf08be6-aea3-4edb-8ed8-345f93b947ef None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:20.120601 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28682cdc-cc94-4386-8eb0-476f9790d26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:20.121206 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28682cdc-cc94-4386-8eb0-476f9790d26a admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:20.121485 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28682cdc-cc94-4386-8eb0-476f9790d26a admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:20.121754 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28682cdc-cc94-4386-8eb0-476f9790d26a admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:20.122453 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28682cdc-cc94-4386-8eb0-476f9790d26a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:20.124336 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28682cdc-cc94-4386-8eb0-476f9790d26a admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:20.152939 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 953/1929] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:20 2022] POST /identity/v3/projects => generated 399 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:20.160306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b463f1-a370-4c3a-b534-8cb41d6c8e25 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:20.172752 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b463f1-a370-4c3a-b534-8cb41d6c8e25 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:20.236388 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca69cf-b467-4817-9cdf-ebe053101925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:20.237054 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca69cf-b467-4817-9cdf-ebe053101925 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:20.237321 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca69cf-b467-4817-9cdf-ebe053101925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:20.237669 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca69cf-b467-4817-9cdf-ebe053101925 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:20.238317 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ca69cf-b467-4817-9cdf-ebe053101925 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:20.240262 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ca69cf-b467-4817-9cdf-ebe053101925 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:20.287517 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 977/1930] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:20 2022] POST /identity/v3/users => generated 300 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:20.291962 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28682cdc-cc94-4386-8eb0-476f9790d26a admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:20.305558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28682cdc-cc94-4386-8eb0-476f9790d26a admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:20.317725 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbe880a-622b-485c-91fd-972a8e37f871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:20.318410 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe880a-622b-485c-91fd-972a8e37f871 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:20.322566 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe880a-622b-485c-91fd-972a8e37f871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:20.322851 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe880a-622b-485c-91fd-972a8e37f871 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:20.323747 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbe880a-622b-485c-91fd-972a8e37f871 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:20.325451 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbe880a-622b-485c-91fd-972a8e37f871 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:20.336838 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 954/1931] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:20.344179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca69cf-b467-4817-9cdf-ebe053101925 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:20.356319 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca69cf-b467-4817-9cdf-ebe053101925 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:20.362140 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a75e39-93ee-4fd7-a40d-736f88373aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:20.362946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a75e39-93ee-4fd7-a40d-736f88373aff admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:20.363425 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a75e39-93ee-4fd7-a40d-736f88373aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:20.363719 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a75e39-93ee-4fd7-a40d-736f88373aff admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:20.364630 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a75e39-93ee-4fd7-a40d-736f88373aff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:20.367444 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a75e39-93ee-4fd7-a40d-736f88373aff admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:20.385505 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 978/1932] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 04:22:20.386626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbe880a-622b-485c-91fd-972a8e37f871 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:20.396960 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbe880a-622b-485c-91fd-972a8e37f871 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:20.401916 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df334b1-c0e1-4300-b5d8-84129b3130e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:20.402628 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df334b1-c0e1-4300-b5d8-84129b3130e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:20.403107 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df334b1-c0e1-4300-b5d8-84129b3130e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:20.403670 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df334b1-c0e1-4300-b5d8-84129b3130e2 admin admin] PATH_INFO: `/v3/projects/37a09d2936ee4cda9d1fe016e604b174/users/5663b9134b7544e088ddebf242f8db7e/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:20.421309 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df334b1-c0e1-4300-b5d8-84129b3130e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37a09d2936ee4cda9d1fe016e604b174, user_id=5663b9134b7544e088ddebf242f8db7e, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:20.424195 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df334b1-c0e1-4300-b5d8-84129b3130e2 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:20.439038 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 955/1933] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:20 2022] PUT /identity/v3/projects/37a09d2936ee4cda9d1fe016e604b174/users/5663b9134b7544e088ddebf242f8db7e/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:22:20.456646 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e119b-ea77-4cde-b25b-f8c7241b039d None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:20.456646 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e119b-ea77-4cde-b25b-f8c7241b039d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:20.456646 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e119b-ea77-4cde-b25b-f8c7241b039d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:20.482633 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-e54e119b-ea77-4cde-b25b-f8c7241b039d None None] MFA Rules not processed for user `5663b9134b7544e088ddebf242f8db7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:20.484573 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e54e119b-ea77-4cde-b25b-f8c7241b039d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:20.565000 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 979/1934] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:20 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:22:20.570139 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df334b1-c0e1-4300-b5d8-84129b3130e2 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:20.650071 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df334b1-c0e1-4300-b5d8-84129b3130e2 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:20.654931 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d914d-1c94-4154-9f29-6c3c0f4af3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:20.655593 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d914d-1c94-4154-9f29-6c3c0f4af3e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:20.655873 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d914d-1c94-4154-9f29-6c3c0f4af3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:20.656150 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d914d-1c94-4154-9f29-6c3c0f4af3e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:20.656850 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5d914d-1c94-4154-9f29-6c3c0f4af3e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:20.658946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5d914d-1c94-4154-9f29-6c3c0f4af3e4 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:20.687270 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 956/1935] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:20 2022] POST /identity/v3/projects => generated 399 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:20.691332 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54e119b-ea77-4cde-b25b-f8c7241b039d None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:20.703004 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54e119b-ea77-4cde-b25b-f8c7241b039d None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:20.738119 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:20.766959 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec227e-4c45-4ce3-b3d5-6ba1c2c0f4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:20.767606 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec227e-4c45-4ce3-b3d5-6ba1c2c0f4c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:20.767891 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec227e-4c45-4ce3-b3d5-6ba1c2c0f4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:20.768164 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec227e-4c45-4ce3-b3d5-6ba1c2c0f4c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:20.768859 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec227e-4c45-4ce3-b3d5-6ba1c2c0f4c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:20.770858 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec227e-4c45-4ce3-b3d5-6ba1c2c0f4c6 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:20.792201 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 980/1936] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:20 2022] POST /identity/v3/users => generated 306 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:20.796106 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d914d-1c94-4154-9f29-6c3c0f4af3e4 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:20.808654 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d914d-1c94-4154-9f29-6c3c0f4af3e4 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:20.813508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed59cc-dba2-4f32-8969-d659f6d33857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:20.814213 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed59cc-dba2-4f32-8969-d659f6d33857 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:20.814500 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed59cc-dba2-4f32-8969-d659f6d33857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:20.814780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed59cc-dba2-4f32-8969-d659f6d33857 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:20.815484 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed59cc-dba2-4f32-8969-d659f6d33857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:20.817093 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed59cc-dba2-4f32-8969-d659f6d33857 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:20.828634 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 957/1937] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:20.832578 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec227e-4c45-4ce3-b3d5-6ba1c2c0f4c6 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:20.843237 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec227e-4c45-4ce3-b3d5-6ba1c2c0f4c6 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:20.847840 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f1114-518b-44cf-96f3-706ff6fb4e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:20.848658 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f1114-518b-44cf-96f3-706ff6fb4e64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:20.849047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f1114-518b-44cf-96f3-706ff6fb4e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:20.849504 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f1114-518b-44cf-96f3-706ff6fb4e64 admin admin] PATH_INFO: `/v3/projects/5ac6fc9debb64c41a5a943c0991c748d/users/ee564943e2da4b0d8134e815a637180a/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:20.864842 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f1114-518b-44cf-96f3-706ff6fb4e64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ac6fc9debb64c41a5a943c0991c748d, user_id=ee564943e2da4b0d8134e815a637180a, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:20.867507 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f1114-518b-44cf-96f3-706ff6fb4e64 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:20.881697 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 981/1938] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:20 2022] PUT /identity/v3/projects/5ac6fc9debb64c41a5a943c0991c748d/users/ee564943e2da4b0d8134e815a637180a/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:20.885873 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed59cc-dba2-4f32-8969-d659f6d33857 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:20.896489 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed59cc-dba2-4f32-8969-d659f6d33857 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:20.951870 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:20.971496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b305a-2b88-439b-b8a7-aad6b815a1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:20.972514 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b305a-2b88-439b-b8a7-aad6b815a1fb admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:20.973015 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b305a-2b88-439b-b8a7-aad6b815a1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:20.973468 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b305a-2b88-439b-b8a7-aad6b815a1fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:20.974222 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068b305a-2b88-439b-b8a7-aad6b815a1fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:20.976256 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068b305a-2b88-439b-b8a7-aad6b815a1fb admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:20.989465 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 958/1939] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:20.992639 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f1114-518b-44cf-96f3-706ff6fb4e64 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:21.005835 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f1114-518b-44cf-96f3-706ff6fb4e64 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:21.016488 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d39500-579e-4252-ab78-3e88ca97df11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:21.017359 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d39500-579e-4252-ab78-3e88ca97df11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:21.017641 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d39500-579e-4252-ab78-3e88ca97df11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:21.017947 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d39500-579e-4252-ab78-3e88ca97df11 admin admin] PATH_INFO: `/v3/projects/5ac6fc9debb64c41a5a943c0991c748d/users/ee564943e2da4b0d8134e815a637180a/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:21.020067 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d39500-579e-4252-ab78-3e88ca97df11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ac6fc9debb64c41a5a943c0991c748d, user_id=ee564943e2da4b0d8134e815a637180a, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:21.022738 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d39500-579e-4252-ab78-3e88ca97df11 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:21.049372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 982/1940] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:20 2022] PUT /identity/v3/projects/5ac6fc9debb64c41a5a943c0991c748d/users/ee564943e2da4b0d8134e815a637180a/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:21.060380 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133ea0a-0a9c-44ce-bcaa-4585ce013925 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:21.060741 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133ea0a-0a9c-44ce-bcaa-4585ce013925 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:21.061114 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133ea0a-0a9c-44ce-bcaa-4585ce013925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:21.097672 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-3133ea0a-0a9c-44ce-bcaa-4585ce013925 None None] MFA Rules not processed for user `ee564943e2da4b0d8134e815a637180a`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:21.098804 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3133ea0a-0a9c-44ce-bcaa-4585ce013925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:21.208896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 959/1941] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:21 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:21.298454 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d39500-579e-4252-ab78-3e88ca97df11 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:21.300302 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-60d39500-579e-4252-ab78-3e88ca97df11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:21.361917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d39500-579e-4252-ab78-3e88ca97df11 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:21.370363 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2864d-0cf9-4ca7-b2dd-6c758845e0b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:21.370363 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2864d-0cf9-4ca7-b2dd-6c758845e0b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:21.370363 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2864d-0cf9-4ca7-b2dd-6c758845e0b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:21.370363 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2864d-0cf9-4ca7-b2dd-6c758845e0b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:21.373587 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3133ea0a-0a9c-44ce-bcaa-4585ce013925 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:21.375053 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3133ea0a-0a9c-44ce-bcaa-4585ce013925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:21.381254 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee2864d-0cf9-4ca7-b2dd-6c758845e0b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:21.382722 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee2864d-0cf9-4ca7-b2dd-6c758845e0b8 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:21.394393 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 983/1942] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:21 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:21.548953 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3133ea0a-0a9c-44ce-bcaa-4585ce013925 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:21.577665 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5923291-d684-4bdb-a3ad-e93ad4962bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:21.579283 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5923291-d684-4bdb-a3ad-e93ad4962bb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:21.579576 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5923291-d684-4bdb-a3ad-e93ad4962bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:21.579857 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5923291-d684-4bdb-a3ad-e93ad4962bb8 admin admin] PATH_INFO: `/v3/users/303ffab8fb7b4b30a1c7e60dd4869f21` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:21.581662 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5923291-d684-4bdb-a3ad-e93ad4962bb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=303ffab8fb7b4b30a1c7e60dd4869f21)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:21.583812 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5923291-d684-4bdb-a3ad-e93ad4962bb8 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:21.592894 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2864d-0cf9-4ca7-b2dd-6c758845e0b8 service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:21.610702 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2864d-0cf9-4ca7-b2dd-6c758845e0b8 service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:21.615302 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59616a97-6cbf-41f6-98bf-766c56ef4c9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:21.615999 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59616a97-6cbf-41f6-98bf-766c56ef4c9c service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:21.616286 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59616a97-6cbf-41f6-98bf-766c56ef4c9c service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:21.616719 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59616a97-6cbf-41f6-98bf-766c56ef4c9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:21.659390 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-e5923291-d684-4bdb-a3ad-e93ad4962bb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '303ffab8fb7b4b30a1c7e60dd4869f21'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:21.660124 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-e5923291-d684-4bdb-a3ad-e93ad4962bb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '303ffab8fb7b4b30a1c7e60dd4869f21'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:21.668994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-e5923291-d684-4bdb-a3ad-e93ad4962bb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '303ffab8fb7b4b30a1c7e60dd4869f21'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:21.686944 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59616a97-6cbf-41f6-98bf-766c56ef4c9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:21.688371 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59616a97-6cbf-41f6-98bf-766c56ef4c9c service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:21.688506 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:21.690462 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-e5923291-d684-4bdb-a3ad-e93ad4962bb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '303ffab8fb7b4b30a1c7e60dd4869f21'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:21.698560 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 984/1943] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:21 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 04:22:21.700746 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-e5923291-d684-4bdb-a3ad-e93ad4962bb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '303ffab8fb7b4b30a1c7e60dd4869f21'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:21.709032 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 960/1944] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:21 2022] DELETE /identity/v3/users/303ffab8fb7b4b30a1c7e60dd4869f21 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:22:21.714028 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba3141-3d6c-4f94-b7ba-bcabb5d5fc83 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:21.714544 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba3141-3d6c-4f94-b7ba-bcabb5d5fc83 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:21.715027 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba3141-3d6c-4f94-b7ba-bcabb5d5fc83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:21.744771 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-41ba3141-3d6c-4f94-b7ba-bcabb5d5fc83 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:21.746461 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-41ba3141-3d6c-4f94-b7ba-bcabb5d5fc83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:21.787614 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:21.834802 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 985/1945] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:21.845732 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5923291-d684-4bdb-a3ad-e93ad4962bb8 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:21.847283 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e5923291-d684-4bdb-a3ad-e93ad4962bb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:21.919271 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5923291-d684-4bdb-a3ad-e93ad4962bb8 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:21.923149 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f68962-3175-43f7-a6c1-b306a3c46a22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:21.923776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f68962-3175-43f7-a6c1-b306a3c46a22 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:21.924055 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f68962-3175-43f7-a6c1-b306a3c46a22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:21.924327 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f68962-3175-43f7-a6c1-b306a3c46a22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:21.934491 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f68962-3175-43f7-a6c1-b306a3c46a22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:21.936121 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f68962-3175-43f7-a6c1-b306a3c46a22 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:21.947052 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 961/1946] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:22.262211 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba3141-3d6c-4f94-b7ba-bcabb5d5fc83 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:22.264190 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-41ba3141-3d6c-4f94-b7ba-bcabb5d5fc83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:22.281390 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba3141-3d6c-4f94-b7ba-bcabb5d5fc83 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:22.287086 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985d1bf-7684-49bf-8798-e5564bf4cd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:22.287985 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985d1bf-7684-49bf-8798-e5564bf4cd86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:22.288460 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985d1bf-7684-49bf-8798-e5564bf4cd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:22.288832 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985d1bf-7684-49bf-8798-e5564bf4cd86 admin admin] PATH_INFO: `/v3/projects/5c69dc7607c94b58b8335101487bb6d8` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:22.290135 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7985d1bf-7684-49bf-8798-e5564bf4cd86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c69dc7607c94b58b8335101487bb6d8)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:22.293114 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7985d1bf-7684-49bf-8798-e5564bf4cd86 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:22.353947 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7985d1bf-7684-49bf-8798-e5564bf4cd86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c69dc7607c94b58b8335101487bb6d8', 'request_id': 'req-7985d1bf-7684-49bf-8798-e5564bf4cd86'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:22.359994 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7985d1bf-7684-49bf-8798-e5564bf4cd86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c69dc7607c94b58b8335101487bb6d8', 'request_id': 'req-7985d1bf-7684-49bf-8798-e5564bf4cd86'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:22.365524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7985d1bf-7684-49bf-8798-e5564bf4cd86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c69dc7607c94b58b8335101487bb6d8', 'request_id': 'req-7985d1bf-7684-49bf-8798-e5564bf4cd86'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:22.379032 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7985d1bf-7684-49bf-8798-e5564bf4cd86 admin admin] The token cache is being invalidate because project 5c69dc7607c94b58b8335101487bb6d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:22.379324 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7985d1bf-7684-49bf-8798-e5564bf4cd86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:22.379959 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7985d1bf-7684-49bf-8798-e5564bf4cd86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:22.381473 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 986/1947] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:22 2022] DELETE /identity/v3/projects/5c69dc7607c94b58b8335101487bb6d8 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:22.386149 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f68962-3175-43f7-a6c1-b306a3c46a22 service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:22.395653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f68962-3175-43f7-a6c1-b306a3c46a22 service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:22.451760 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a019ef1-7833-49c8-9b1f-f9b8f6e315c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:22.452405 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a019ef1-7833-49c8-9b1f-f9b8f6e315c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:22.452681 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a019ef1-7833-49c8-9b1f-f9b8f6e315c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:22.452957 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a019ef1-7833-49c8-9b1f-f9b8f6e315c5 admin admin] PATH_INFO: `/v3/users/6ab6637397064e1b82c9ecebe3d2aa01` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:22.454153 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a019ef1-7833-49c8-9b1f-f9b8f6e315c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ab6637397064e1b82c9ecebe3d2aa01)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:22.456066 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a019ef1-7833-49c8-9b1f-f9b8f6e315c5 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:22.501453 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0a019ef1-7833-49c8-9b1f-f9b8f6e315c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ab6637397064e1b82c9ecebe3d2aa01'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:22.502143 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0a019ef1-7833-49c8-9b1f-f9b8f6e315c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ab6637397064e1b82c9ecebe3d2aa01'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:22.510215 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0a019ef1-7833-49c8-9b1f-f9b8f6e315c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ab6637397064e1b82c9ecebe3d2aa01'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:22.522164 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0a019ef1-7833-49c8-9b1f-f9b8f6e315c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ab6637397064e1b82c9ecebe3d2aa01'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:22.529174 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0a019ef1-7833-49c8-9b1f-f9b8f6e315c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ab6637397064e1b82c9ecebe3d2aa01'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:22.535024 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 962/1948] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:22 2022] DELETE /identity/v3/users/6ab6637397064e1b82c9ecebe3d2aa01 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:22:22.815185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985d1bf-7684-49bf-8798-e5564bf4cd86 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:22.817011 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7985d1bf-7684-49bf-8798-e5564bf4cd86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:22.830682 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985d1bf-7684-49bf-8798-e5564bf4cd86 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:22.889178 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d51bdc-eb89-4ae5-809c-da26d876b46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:22.889764 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d51bdc-eb89-4ae5-809c-da26d876b46f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:22.890081 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d51bdc-eb89-4ae5-809c-da26d876b46f admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:22.890362 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d51bdc-eb89-4ae5-809c-da26d876b46f admin admin] PATH_INFO: `/v3/projects/8940150716564661aec86b7727fdfcc3` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:22.891534 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d51bdc-eb89-4ae5-809c-da26d876b46f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8940150716564661aec86b7727fdfcc3)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:22.893442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d51bdc-eb89-4ae5-809c-da26d876b46f admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:22.939746 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-60d51bdc-eb89-4ae5-809c-da26d876b46f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8940150716564661aec86b7727fdfcc3', 'request_id': 'req-60d51bdc-eb89-4ae5-809c-da26d876b46f'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:22.946055 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-60d51bdc-eb89-4ae5-809c-da26d876b46f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8940150716564661aec86b7727fdfcc3', 'request_id': 'req-60d51bdc-eb89-4ae5-809c-da26d876b46f'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:22.947918 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:22.953372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-60d51bdc-eb89-4ae5-809c-da26d876b46f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8940150716564661aec86b7727fdfcc3', 'request_id': 'req-60d51bdc-eb89-4ae5-809c-da26d876b46f'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:22.967923 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-60d51bdc-eb89-4ae5-809c-da26d876b46f admin admin] The token cache is being invalidate because project 8940150716564661aec86b7727fdfcc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:22.968211 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-60d51bdc-eb89-4ae5-809c-da26d876b46f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:22.968833 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-60d51bdc-eb89-4ae5-809c-da26d876b46f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:22.970368 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 987/1949] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:22 2022] DELETE /identity/v3/projects/8940150716564661aec86b7727fdfcc3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:22.975210 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc28760-6978-4cd1-84a9-f4f7f3596d1f None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:22.975526 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc28760-6978-4cd1-84a9-f4f7f3596d1f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:22.975806 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc28760-6978-4cd1-84a9-f4f7f3596d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:22.996028 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-5fc28760-6978-4cd1-84a9-f4f7f3596d1f None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:22.997216 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-5fc28760-6978-4cd1-84a9-f4f7f3596d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:23.001435 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:23.065940 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 963/1950] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:22:23.074351 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978470b-d4f1-4830-850a-240a826755e1 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:23.074640 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978470b-d4f1-4830-850a-240a826755e1 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:23.074920 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978470b-d4f1-4830-850a-240a826755e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:23.096592 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-d978470b-d4f1-4830-850a-240a826755e1 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:23.097817 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d978470b-d4f1-4830-850a-240a826755e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:23.106639 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 988/1951] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:23.111707 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc28760-6978-4cd1-84a9-f4f7f3596d1f None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:23.121632 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc28760-6978-4cd1-84a9-f4f7f3596d1f None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:23.126287 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408fdf5-b56c-4952-b851-d7884826be9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:23.126901 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408fdf5-b56c-4952-b851-d7884826be9a admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:23.127187 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408fdf5-b56c-4952-b851-d7884826be9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:23.127464 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408fdf5-b56c-4952-b851-d7884826be9a admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:23.128169 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f408fdf5-b56c-4952-b851-d7884826be9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:22:23.128479 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f408fdf5-b56c-4952-b851-d7884826be9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:23.130113 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f408fdf5-b56c-4952-b851-d7884826be9a admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:23.142359 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 964/1952] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:22:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:22:23.147397 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978470b-d4f1-4830-850a-240a826755e1 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:23.157769 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978470b-d4f1-4830-850a-240a826755e1 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:23.162618 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e3d21b-6013-43ae-a904-6e863eebedde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:23.163257 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3d21b-6013-43ae-a904-6e863eebedde admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:23.163529 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3d21b-6013-43ae-a904-6e863eebedde admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:23.163802 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3d21b-6013-43ae-a904-6e863eebedde admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:23.164506 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e3d21b-6013-43ae-a904-6e863eebedde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:23.166486 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e3d21b-6013-43ae-a904-6e863eebedde admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:23.192906 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 989/1953] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:23 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:23.197934 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408fdf5-b56c-4952-b851-d7884826be9a admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:23.207845 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408fdf5-b56c-4952-b851-d7884826be9a admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:23.266112 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9ca53-9c73-482a-9602-dd4e1de76561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:23.266732 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9ca53-9c73-482a-9602-dd4e1de76561 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:23.267010 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9ca53-9c73-482a-9602-dd4e1de76561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:23.267281 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9ca53-9c73-482a-9602-dd4e1de76561 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:23.267956 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9ca53-9c73-482a-9602-dd4e1de76561 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:23.269861 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9ca53-9c73-482a-9602-dd4e1de76561 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:23.294149 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 965/1954] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:23 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:22:23.295382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e3d21b-6013-43ae-a904-6e863eebedde admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:23.305434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e3d21b-6013-43ae-a904-6e863eebedde admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:23.310834 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cce633-89e3-4b7e-a4e0-402207a7c6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:23.311474 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cce633-89e3-4b7e-a4e0-402207a7c6ba admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:23.311755 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cce633-89e3-4b7e-a4e0-402207a7c6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:23.312022 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cce633-89e3-4b7e-a4e0-402207a7c6ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:23.312720 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cce633-89e3-4b7e-a4e0-402207a7c6ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:23.314353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cce633-89e3-4b7e-a4e0-402207a7c6ba admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:23.325045 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 990/1955] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 04:22:23.328933 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9ca53-9c73-482a-9602-dd4e1de76561 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:23.339018 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9ca53-9c73-482a-9602-dd4e1de76561 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:23.343595 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44a5fe-dcb3-41d7-8b1e-917ab1070058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:23.344205 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44a5fe-dcb3-41d7-8b1e-917ab1070058 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:23.344486 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44a5fe-dcb3-41d7-8b1e-917ab1070058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:23.344761 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44a5fe-dcb3-41d7-8b1e-917ab1070058 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:23.345460 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d44a5fe-dcb3-41d7-8b1e-917ab1070058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:23.347072 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d44a5fe-dcb3-41d7-8b1e-917ab1070058 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:23.358007 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 966/1956] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:23.362318 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cce633-89e3-4b7e-a4e0-402207a7c6ba admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:23.380624 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cce633-89e3-4b7e-a4e0-402207a7c6ba admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:23.387351 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404873e5-d1ef-49a7-9f2a-255b5df5d671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:23.388132 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404873e5-d1ef-49a7-9f2a-255b5df5d671 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:23.388312 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404873e5-d1ef-49a7-9f2a-255b5df5d671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:23.388597 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404873e5-d1ef-49a7-9f2a-255b5df5d671 admin admin] PATH_INFO: `/v3/projects/b8db8424e931469bb555033d8939b450/users/de47f559667a49de8cb8b42109c9a6c1/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:23.404446 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404873e5-d1ef-49a7-9f2a-255b5df5d671 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8db8424e931469bb555033d8939b450, user_id=de47f559667a49de8cb8b42109c9a6c1, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:23.407177 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404873e5-d1ef-49a7-9f2a-255b5df5d671 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:23.422900 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 991/1957] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:23 2022] PUT /identity/v3/projects/b8db8424e931469bb555033d8939b450/users/de47f559667a49de8cb8b42109c9a6c1/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:23.439860 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab42910-7a8f-43d3-b857-004e7a41ce9d None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:23.440390 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab42910-7a8f-43d3-b857-004e7a41ce9d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:23.440390 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab42910-7a8f-43d3-b857-004e7a41ce9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:23.472762 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-2ab42910-7a8f-43d3-b857-004e7a41ce9d None None] MFA Rules not processed for user `de47f559667a49de8cb8b42109c9a6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:23.473960 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2ab42910-7a8f-43d3-b857-004e7a41ce9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:23.538720 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 967/1958] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:23.543261 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404873e5-d1ef-49a7-9f2a-255b5df5d671 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:23.605396 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404873e5-d1ef-49a7-9f2a-255b5df5d671 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:23.609927 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c17f263-631b-42a2-bb3d-164631c32f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:23.610565 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17f263-631b-42a2-bb3d-164631c32f9e admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:23.610843 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17f263-631b-42a2-bb3d-164631c32f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:23.611114 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17f263-631b-42a2-bb3d-164631c32f9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:23.611782 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c17f263-631b-42a2-bb3d-164631c32f9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:23.613660 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c17f263-631b-42a2-bb3d-164631c32f9e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:23.640995 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 992/1959] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:23 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:23.645569 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab42910-7a8f-43d3-b857-004e7a41ce9d None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:23.653018 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:23.657272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab42910-7a8f-43d3-b857-004e7a41ce9d None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:23.717391 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f74a5-a1f9-445e-b149-5b9007f279dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:23.718055 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f74a5-a1f9-445e-b149-5b9007f279dd admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:23.718342 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f74a5-a1f9-445e-b149-5b9007f279dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:23.718623 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f74a5-a1f9-445e-b149-5b9007f279dd admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:23.719301 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f74a5-a1f9-445e-b149-5b9007f279dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:23.721218 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f74a5-a1f9-445e-b149-5b9007f279dd admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:23.742428 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 968/1960] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:23 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:23.746926 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c17f263-631b-42a2-bb3d-164631c32f9e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:23.756972 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c17f263-631b-42a2-bb3d-164631c32f9e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:23.761630 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e2de90-3088-4d5b-b2d4-87cc80940518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:23.762267 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e2de90-3088-4d5b-b2d4-87cc80940518 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:23.762548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e2de90-3088-4d5b-b2d4-87cc80940518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:23.762820 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e2de90-3088-4d5b-b2d4-87cc80940518 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:23.763513 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e2de90-3088-4d5b-b2d4-87cc80940518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:23.765087 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e2de90-3088-4d5b-b2d4-87cc80940518 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:23.775363 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 993/1961] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:23.779446 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f74a5-a1f9-445e-b149-5b9007f279dd admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:23.789256 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f74a5-a1f9-445e-b149-5b9007f279dd admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:23.796597 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5777a5d-b6d5-4715-8415-c605be026063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:23.797328 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5777a5d-b6d5-4715-8415-c605be026063 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:23.797614 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5777a5d-b6d5-4715-8415-c605be026063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:23.797933 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5777a5d-b6d5-4715-8415-c605be026063 admin admin] PATH_INFO: `/v3/projects/38f8b2cbead2496c907255bb7830779f/users/af5cc3b5758545c4bcf0e4d995296a89/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:23.817862 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5777a5d-b6d5-4715-8415-c605be026063 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38f8b2cbead2496c907255bb7830779f, user_id=af5cc3b5758545c4bcf0e4d995296a89, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:23.820956 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5777a5d-b6d5-4715-8415-c605be026063 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:23.834299 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 969/1962] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:23 2022] PUT /identity/v3/projects/38f8b2cbead2496c907255bb7830779f/users/af5cc3b5758545c4bcf0e4d995296a89/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:23.838783 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e2de90-3088-4d5b-b2d4-87cc80940518 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:23.849104 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e2de90-3088-4d5b-b2d4-87cc80940518 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:23.900169 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:23.910400 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdbe70d-341c-4adf-86fd-f31861cf0e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:23.911050 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdbe70d-341c-4adf-86fd-f31861cf0e03 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:23.911774 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdbe70d-341c-4adf-86fd-f31861cf0e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:23.912254 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdbe70d-341c-4adf-86fd-f31861cf0e03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:23.913223 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdbe70d-341c-4adf-86fd-f31861cf0e03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:23.914937 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdbe70d-341c-4adf-86fd-f31861cf0e03 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:23.929007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 994/1963] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:23 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:23.930204 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5777a5d-b6d5-4715-8415-c605be026063 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:23.941024 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5777a5d-b6d5-4715-8415-c605be026063 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:23.945809 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598aaf51-648e-41d9-b118-15f6af9172c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:23.946523 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598aaf51-648e-41d9-b118-15f6af9172c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:23.946806 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598aaf51-648e-41d9-b118-15f6af9172c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:23.947078 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598aaf51-648e-41d9-b118-15f6af9172c5 admin admin] PATH_INFO: `/v3/projects/38f8b2cbead2496c907255bb7830779f/users/af5cc3b5758545c4bcf0e4d995296a89/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:23.949490 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598aaf51-648e-41d9-b118-15f6af9172c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38f8b2cbead2496c907255bb7830779f, user_id=af5cc3b5758545c4bcf0e4d995296a89, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:23.952146 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598aaf51-648e-41d9-b118-15f6af9172c5 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:23.962846 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 970/1964] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:23 2022] PUT /identity/v3/projects/38f8b2cbead2496c907255bb7830779f/users/af5cc3b5758545c4bcf0e4d995296a89/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:22:23.974451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0cce0-7c96-458a-b833-9e49838caeb3 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:23.974734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0cce0-7c96-458a-b833-9e49838caeb3 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:23.975007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0cce0-7c96-458a-b833-9e49838caeb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:24.003716 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-1fb0cce0-7c96-458a-b833-9e49838caeb3 None None] MFA Rules not processed for user `af5cc3b5758545c4bcf0e4d995296a89`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:24.004929 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-1fb0cce0-7c96-458a-b833-9e49838caeb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:24.076091 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 995/1965] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:23 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:22:24.087074 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598aaf51-648e-41d9-b118-15f6af9172c5 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:24.089416 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-598aaf51-648e-41d9-b118-15f6af9172c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:24.180188 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598aaf51-648e-41d9-b118-15f6af9172c5 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:24.184201 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a0c3e3-0856-46c8-b4f6-ba67f01dcd7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:24.184900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0c3e3-0856-46c8-b4f6-ba67f01dcd7b service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:24.185194 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0c3e3-0856-46c8-b4f6-ba67f01dcd7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:24.185464 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0c3e3-0856-46c8-b4f6-ba67f01dcd7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:24.195281 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a0c3e3-0856-46c8-b4f6-ba67f01dcd7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:24.196753 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a0c3e3-0856-46c8-b4f6-ba67f01dcd7b service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:24.206792 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 971/1966] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:24 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:24.691070 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0cce0-7c96-458a-b833-9e49838caeb3 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:24.771960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0cce0-7c96-458a-b833-9e49838caeb3 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:24.778362 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca97c63-cac2-4610-ba9f-617c2ea53621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:24.779081 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca97c63-cac2-4610-ba9f-617c2ea53621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:24.779354 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca97c63-cac2-4610-ba9f-617c2ea53621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:24.779632 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca97c63-cac2-4610-ba9f-617c2ea53621 admin admin] PATH_INFO: `/v3/users/de47f559667a49de8cb8b42109c9a6c1` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:24.780901 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca97c63-cac2-4610-ba9f-617c2ea53621 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de47f559667a49de8cb8b42109c9a6c1)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:24.783033 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca97c63-cac2-4610-ba9f-617c2ea53621 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:24.855465 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3ca97c63-cac2-4610-ba9f-617c2ea53621 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de47f559667a49de8cb8b42109c9a6c1'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:24.856166 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3ca97c63-cac2-4610-ba9f-617c2ea53621 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de47f559667a49de8cb8b42109c9a6c1'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:24.856837 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3ca97c63-cac2-4610-ba9f-617c2ea53621 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de47f559667a49de8cb8b42109c9a6c1'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:24.871086 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3ca97c63-cac2-4610-ba9f-617c2ea53621 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de47f559667a49de8cb8b42109c9a6c1'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:24.883766 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3ca97c63-cac2-4610-ba9f-617c2ea53621 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de47f559667a49de8cb8b42109c9a6c1'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:24.887432 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:24.905259 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 996/1967] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:24 2022] DELETE /identity/v3/users/de47f559667a49de8cb8b42109c9a6c1 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:24.910626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16795cb4-443d-45b9-bc40-42f00211826b None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:24.911151 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16795cb4-443d-45b9-bc40-42f00211826b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:24.911462 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16795cb4-443d-45b9-bc40-42f00211826b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:24.933385 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-16795cb4-443d-45b9-bc40-42f00211826b None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:24.935049 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-16795cb4-443d-45b9-bc40-42f00211826b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:25.000860 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:25.012895 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 972/1968] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:25.024159 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca97c63-cac2-4610-ba9f-617c2ea53621 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:25.026319 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-3ca97c63-cac2-4610-ba9f-617c2ea53621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:25.097902 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca97c63-cac2-4610-ba9f-617c2ea53621 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:25.101538 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8db076-ca84-4202-b490-f308fad93c82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:25.102319 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8db076-ca84-4202-b490-f308fad93c82 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:25.102707 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8db076-ca84-4202-b490-f308fad93c82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:25.103138 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8db076-ca84-4202-b490-f308fad93c82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:25.115836 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8db076-ca84-4202-b490-f308fad93c82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:25.117502 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8db076-ca84-4202-b490-f308fad93c82 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:25.130906 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 997/1969] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:25.372579 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16795cb4-443d-45b9-bc40-42f00211826b None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:25.372579 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-16795cb4-443d-45b9-bc40-42f00211826b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:25.392092 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16795cb4-443d-45b9-bc40-42f00211826b None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:25.406661 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd1fbf8-1a8e-438e-93d7-c0039fe831da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:25.407314 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1fbf8-1a8e-438e-93d7-c0039fe831da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:25.407559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1fbf8-1a8e-438e-93d7-c0039fe831da admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:25.407841 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1fbf8-1a8e-438e-93d7-c0039fe831da admin admin] PATH_INFO: `/v3/users/e71d2e81acb048beb116c8b31fe68a56` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:25.410731 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd1fbf8-1a8e-438e-93d7-c0039fe831da admin admin] RBAC: Authorizing `identity:delete_user(user_id=e71d2e81acb048beb116c8b31fe68a56)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:25.412939 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd1fbf8-1a8e-438e-93d7-c0039fe831da admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:25.523864 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4dd1fbf8-1a8e-438e-93d7-c0039fe831da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e71d2e81acb048beb116c8b31fe68a56'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:25.524679 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4dd1fbf8-1a8e-438e-93d7-c0039fe831da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e71d2e81acb048beb116c8b31fe68a56'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:25.544732 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4dd1fbf8-1a8e-438e-93d7-c0039fe831da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e71d2e81acb048beb116c8b31fe68a56'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:25.573538 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4dd1fbf8-1a8e-438e-93d7-c0039fe831da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e71d2e81acb048beb116c8b31fe68a56'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:25.594783 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4dd1fbf8-1a8e-438e-93d7-c0039fe831da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e71d2e81acb048beb116c8b31fe68a56'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:25.603772 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 973/1970] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:25 2022] DELETE /identity/v3/users/e71d2e81acb048beb116c8b31fe68a56 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:22:25.608795 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db9109-5d55-4600-8956-b592594ef102 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:25.609063 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db9109-5d55-4600-8956-b592594ef102 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:25.609457 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db9109-5d55-4600-8956-b592594ef102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:25.641589 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-09db9109-5d55-4600-8956-b592594ef102 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:25.642949 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-09db9109-5d55-4600-8956-b592594ef102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:25.648664 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd1fbf8-1a8e-438e-93d7-c0039fe831da admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:25.651426 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-4dd1fbf8-1a8e-438e-93d7-c0039fe831da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:25.758769 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 998/1971] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:25.763590 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd1fbf8-1a8e-438e-93d7-c0039fe831da admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:25.765305 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db9109-5d55-4600-8956-b592594ef102 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:25.768000 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-09db9109-5d55-4600-8956-b592594ef102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:25.768967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1c6b8-948e-44f1-9a7f-5a5bd78f7dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:25.770018 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1c6b8-948e-44f1-9a7f-5a5bd78f7dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:25.770460 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1c6b8-948e-44f1-9a7f-5a5bd78f7dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:25.770834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1c6b8-948e-44f1-9a7f-5a5bd78f7dd5 admin admin] PATH_INFO: `/v3/projects/b8db8424e931469bb555033d8939b450` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:25.772272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce1c6b8-948e-44f1-9a7f-5a5bd78f7dd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8db8424e931469bb555033d8939b450)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:25.783237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce1c6b8-948e-44f1-9a7f-5a5bd78f7dd5 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:25.841714 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0ce1c6b8-948e-44f1-9a7f-5a5bd78f7dd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8db8424e931469bb555033d8939b450', 'request_id': 'req-0ce1c6b8-948e-44f1-9a7f-5a5bd78f7dd5'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:25.846765 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db9109-5d55-4600-8956-b592594ef102 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:25.849276 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0ce1c6b8-948e-44f1-9a7f-5a5bd78f7dd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8db8424e931469bb555033d8939b450', 'request_id': 'req-0ce1c6b8-948e-44f1-9a7f-5a5bd78f7dd5'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:25.857743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0ce1c6b8-948e-44f1-9a7f-5a5bd78f7dd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8db8424e931469bb555033d8939b450', 'request_id': 'req-0ce1c6b8-948e-44f1-9a7f-5a5bd78f7dd5'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:25.874613 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0ce1c6b8-948e-44f1-9a7f-5a5bd78f7dd5 admin admin] The token cache is being invalidate because project b8db8424e931469bb555033d8939b450 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:25.874896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0ce1c6b8-948e-44f1-9a7f-5a5bd78f7dd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:25.875602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0ce1c6b8-948e-44f1-9a7f-5a5bd78f7dd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:25.877669 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 974/1972] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:25 2022] DELETE /identity/v3/projects/b8db8424e931469bb555033d8939b450 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:25.881482 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1c6b8-948e-44f1-9a7f-5a5bd78f7dd5 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:25.883005 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0ce1c6b8-948e-44f1-9a7f-5a5bd78f7dd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:25.888442 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:25.910723 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077684b-34bc-4a2f-8d79-55392f245a31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:25.911585 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077684b-34bc-4a2f-8d79-55392f245a31 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:25.911987 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077684b-34bc-4a2f-8d79-55392f245a31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:25.912274 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077684b-34bc-4a2f-8d79-55392f245a31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:25.914257 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c077684b-34bc-4a2f-8d79-55392f245a31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:25.921498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:25.966264 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1c6b8-948e-44f1-9a7f-5a5bd78f7dd5 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:25.972001 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fec74f-93bd-48f1-88c1-96f2ad0f69d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:25.972661 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fec74f-93bd-48f1-88c1-96f2ad0f69d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:25.972940 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fec74f-93bd-48f1-88c1-96f2ad0f69d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:25.973373 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fec74f-93bd-48f1-88c1-96f2ad0f69d2 admin admin] PATH_INFO: `/v3/users/af5cc3b5758545c4bcf0e4d995296a89` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:25.974750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fec74f-93bd-48f1-88c1-96f2ad0f69d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af5cc3b5758545c4bcf0e4d995296a89)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:25.977333 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fec74f-93bd-48f1-88c1-96f2ad0f69d2 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:25.986535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c077684b-34bc-4a2f-8d79-55392f245a31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:25.988083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c077684b-34bc-4a2f-8d79-55392f245a31 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:26.009929 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 999/1973] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:26.040178 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-30fec74f-93bd-48f1-88c1-96f2ad0f69d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af5cc3b5758545c4bcf0e4d995296a89'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:26.040925 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-30fec74f-93bd-48f1-88c1-96f2ad0f69d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af5cc3b5758545c4bcf0e4d995296a89'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:26.049016 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-30fec74f-93bd-48f1-88c1-96f2ad0f69d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af5cc3b5758545c4bcf0e4d995296a89'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:26.069357 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-30fec74f-93bd-48f1-88c1-96f2ad0f69d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af5cc3b5758545c4bcf0e4d995296a89'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:26.079617 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-30fec74f-93bd-48f1-88c1-96f2ad0f69d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af5cc3b5758545c4bcf0e4d995296a89'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:26.087987 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 975/1974] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:25 2022] DELETE /identity/v3/users/af5cc3b5758545c4bcf0e4d995296a89 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:22:26.442273 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077684b-34bc-4a2f-8d79-55392f245a31 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:26.447009 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c077684b-34bc-4a2f-8d79-55392f245a31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:26.457787 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fec74f-93bd-48f1-88c1-96f2ad0f69d2 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:26.459849 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-30fec74f-93bd-48f1-88c1-96f2ad0f69d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:26.483486 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077684b-34bc-4a2f-8d79-55392f245a31 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:26.540789 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fec74f-93bd-48f1-88c1-96f2ad0f69d2 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:26.546174 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff543a04-c2cc-4247-a699-e15f29a2ce69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:26.546830 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff543a04-c2cc-4247-a699-e15f29a2ce69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:26.547114 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff543a04-c2cc-4247-a699-e15f29a2ce69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:26.547393 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff543a04-c2cc-4247-a699-e15f29a2ce69 admin admin] PATH_INFO: `/v3/projects/38f8b2cbead2496c907255bb7830779f` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:26.548589 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff543a04-c2cc-4247-a699-e15f29a2ce69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38f8b2cbead2496c907255bb7830779f)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:26.550193 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1ca99-59d7-46db-83ab-90f5ab3470f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:26.550583 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff543a04-c2cc-4247-a699-e15f29a2ce69 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:26.550846 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1ca99-59d7-46db-83ab-90f5ab3470f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:26.551131 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1ca99-59d7-46db-83ab-90f5ab3470f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:26.551409 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1ca99-59d7-46db-83ab-90f5ab3470f0 admin admin] PATH_INFO: `/v3/projects/ba694613897b4442bf3a82f72a75a8b3` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:26.552702 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b1ca99-59d7-46db-83ab-90f5ab3470f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba694613897b4442bf3a82f72a75a8b3)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:26.554978 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b1ca99-59d7-46db-83ab-90f5ab3470f0 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:26.577189 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:26.606243 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ff543a04-c2cc-4247-a699-e15f29a2ce69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38f8b2cbead2496c907255bb7830779f', 'request_id': 'req-ff543a04-c2cc-4247-a699-e15f29a2ce69'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:26.609251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b7b1ca99-59d7-46db-83ab-90f5ab3470f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba694613897b4442bf3a82f72a75a8b3', 'request_id': 'req-b7b1ca99-59d7-46db-83ab-90f5ab3470f0'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:26.613277 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ff543a04-c2cc-4247-a699-e15f29a2ce69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38f8b2cbead2496c907255bb7830779f', 'request_id': 'req-ff543a04-c2cc-4247-a699-e15f29a2ce69'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:26.616767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b7b1ca99-59d7-46db-83ab-90f5ab3470f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba694613897b4442bf3a82f72a75a8b3', 'request_id': 'req-b7b1ca99-59d7-46db-83ab-90f5ab3470f0'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:26.619329 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ff543a04-c2cc-4247-a699-e15f29a2ce69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38f8b2cbead2496c907255bb7830779f', 'request_id': 'req-ff543a04-c2cc-4247-a699-e15f29a2ce69'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:26.623579 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b7b1ca99-59d7-46db-83ab-90f5ab3470f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba694613897b4442bf3a82f72a75a8b3', 'request_id': 'req-b7b1ca99-59d7-46db-83ab-90f5ab3470f0'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:26.637479 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b7b1ca99-59d7-46db-83ab-90f5ab3470f0 admin admin] The token cache is being invalidate because project ba694613897b4442bf3a82f72a75a8b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:26.637916 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b7b1ca99-59d7-46db-83ab-90f5ab3470f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:26.638631 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b7b1ca99-59d7-46db-83ab-90f5ab3470f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:26.640214 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1000/1975] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:26 2022] DELETE /identity/v3/projects/ba694613897b4442bf3a82f72a75a8b3 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:26.641601 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ff543a04-c2cc-4247-a699-e15f29a2ce69 admin admin] The token cache is being invalidate because project 38f8b2cbead2496c907255bb7830779f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:26.641896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ff543a04-c2cc-4247-a699-e15f29a2ce69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:26.642567 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ff543a04-c2cc-4247-a699-e15f29a2ce69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:26.644299 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 976/1976] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:26 2022] DELETE /identity/v3/projects/38f8b2cbead2496c907255bb7830779f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:26.644409 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1ca99-59d7-46db-83ab-90f5ab3470f0 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:26.645694 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b7b1ca99-59d7-46db-83ab-90f5ab3470f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:26.648708 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dfbfd8-6ddb-4f5c-a921-25e2867c372a None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:26.648993 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dfbfd8-6ddb-4f5c-a921-25e2867c372a None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:26.649270 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dfbfd8-6ddb-4f5c-a921-25e2867c372a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:26.672150 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-51dfbfd8-6ddb-4f5c-a921-25e2867c372a None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:26.673365 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-51dfbfd8-6ddb-4f5c-a921-25e2867c372a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:26.682285 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:26.717149 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1ca99-59d7-46db-83ab-90f5ab3470f0 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:26.722519 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e7fee-b079-46ea-8f25-9c2711a6482f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:26.723176 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e7fee-b079-46ea-8f25-9c2711a6482f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:26.723461 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e7fee-b079-46ea-8f25-9c2711a6482f admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:26.723733 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e7fee-b079-46ea-8f25-9c2711a6482f admin admin] PATH_INFO: `/v3/users/8dd1d8478f484d2191dbf8188abd4427` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:26.724918 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e7fee-b079-46ea-8f25-9c2711a6482f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dd1d8478f484d2191dbf8188abd4427)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:26.726999 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e7fee-b079-46ea-8f25-9c2711a6482f admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:26.741665 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 977/1977] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:26.750173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9de4c-98ad-4e14-8440-ba0db44030ed None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:26.750450 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9de4c-98ad-4e14-8440-ba0db44030ed None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:26.750725 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9de4c-98ad-4e14-8440-ba0db44030ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:26.772540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-70c9de4c-98ad-4e14-8440-ba0db44030ed None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:26.773257 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-70c9de4c-98ad-4e14-8440-ba0db44030ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:26.781616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1e2e7fee-b079-46ea-8f25-9c2711a6482f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dd1d8478f484d2191dbf8188abd4427'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:26.782379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1e2e7fee-b079-46ea-8f25-9c2711a6482f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dd1d8478f484d2191dbf8188abd4427'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:26.783114 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1e2e7fee-b079-46ea-8f25-9c2711a6482f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dd1d8478f484d2191dbf8188abd4427'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:26.785095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 978/1978] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:22:26.788959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c9de4c-98ad-4e14-8440-ba0db44030ed None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:26.799503 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c9de4c-98ad-4e14-8440-ba0db44030ed None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:26.804539 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb934a-c6a1-461d-9934-6d42f5a2cfde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:26.805222 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb934a-c6a1-461d-9934-6d42f5a2cfde admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:26.805514 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb934a-c6a1-461d-9934-6d42f5a2cfde admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:26.805780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb934a-c6a1-461d-9934-6d42f5a2cfde admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:26.807107 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb934a-c6a1-461d-9934-6d42f5a2cfde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:22:26.807567 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb934a-c6a1-461d-9934-6d42f5a2cfde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:26.809150 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb934a-c6a1-461d-9934-6d42f5a2cfde admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:26.815933 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1e2e7fee-b079-46ea-8f25-9c2711a6482f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dd1d8478f484d2191dbf8188abd4427'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:26.822880 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 979/1979] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:22:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:22:26.826934 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb934a-c6a1-461d-9934-6d42f5a2cfde admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:26.829473 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c2fb934a-c6a1-461d-9934-6d42f5a2cfde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:26.832171 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1e2e7fee-b079-46ea-8f25-9c2711a6482f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dd1d8478f484d2191dbf8188abd4427'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:26.849153 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb934a-c6a1-461d-9934-6d42f5a2cfde admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:26.854950 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452a9a3-f9fe-40eb-befc-49bb55fc749c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:26.855573 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452a9a3-f9fe-40eb-befc-49bb55fc749c admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:26.855852 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452a9a3-f9fe-40eb-befc-49bb55fc749c admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:26.856123 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452a9a3-f9fe-40eb-befc-49bb55fc749c admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:26.856790 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0452a9a3-f9fe-40eb-befc-49bb55fc749c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:26.858325 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1001/1980] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:26 2022] DELETE /identity/v3/users/8dd1d8478f484d2191dbf8188abd4427 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:22:26.862113 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0452a9a3-f9fe-40eb-befc-49bb55fc749c admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:26.895200 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 980/1981] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:26 2022] POST /identity/v3/projects => generated 377 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:22:26.896730 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e7fee-b079-46ea-8f25-9c2711a6482f admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:26.993185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e7fee-b079-46ea-8f25-9c2711a6482f admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:27.079606 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08833387-022c-4e60-a57d-fa2c4e60f1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:27.080323 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08833387-022c-4e60-a57d-fa2c4e60f1af admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:27.081066 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08833387-022c-4e60-a57d-fa2c4e60f1af admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:27.081066 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08833387-022c-4e60-a57d-fa2c4e60f1af admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:27.081817 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08833387-022c-4e60-a57d-fa2c4e60f1af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:27.084727 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08833387-022c-4e60-a57d-fa2c4e60f1af admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:27.093492 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:27.113828 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1002/1982] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:26 2022] POST /identity/v3/users => generated 289 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:27.114660 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452a9a3-f9fe-40eb-befc-49bb55fc749c admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:27.127008 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452a9a3-f9fe-40eb-befc-49bb55fc749c admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:27.132162 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa4ea4-7537-44cd-a0a4-bcf0acdfd8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:27.133505 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa4ea4-7537-44cd-a0a4-bcf0acdfd8bd admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:27.133917 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa4ea4-7537-44cd-a0a4-bcf0acdfd8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:27.135161 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa4ea4-7537-44cd-a0a4-bcf0acdfd8bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:27.138670 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa4ea4-7537-44cd-a0a4-bcf0acdfd8bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:27.138670 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa4ea4-7537-44cd-a0a4-bcf0acdfd8bd admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:27.155196 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08833387-022c-4e60-a57d-fa2c4e60f1af admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:27.155735 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 981/1983] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 04:22:27.166810 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08833387-022c-4e60-a57d-fa2c4e60f1af admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:27.171670 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9194cf3-a39d-4165-af6d-0425c63a1bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:27.172306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9194cf3-a39d-4165-af6d-0425c63a1bec admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:27.172588 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9194cf3-a39d-4165-af6d-0425c63a1bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:27.172866 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9194cf3-a39d-4165-af6d-0425c63a1bec admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:27.174320 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9194cf3-a39d-4165-af6d-0425c63a1bec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:27.175207 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9194cf3-a39d-4165-af6d-0425c63a1bec admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:27.191397 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1003/1984] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:27.192574 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa4ea4-7537-44cd-a0a4-bcf0acdfd8bd admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:27.205555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa4ea4-7537-44cd-a0a4-bcf0acdfd8bd admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:27.210531 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b87cc-a5ab-4913-ad17-2ce133124e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:27.211308 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b87cc-a5ab-4913-ad17-2ce133124e93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:27.211583 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b87cc-a5ab-4913-ad17-2ce133124e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:27.211859 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b87cc-a5ab-4913-ad17-2ce133124e93 admin admin] PATH_INFO: `/v3/projects/6365712030d540bca7469ed29041ee74/users/24b5ec62da9645a9bf28a7f6e38b2cf2/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:27.226868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151b87cc-a5ab-4913-ad17-2ce133124e93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6365712030d540bca7469ed29041ee74, user_id=24b5ec62da9645a9bf28a7f6e38b2cf2, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:27.229580 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151b87cc-a5ab-4913-ad17-2ce133124e93 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:27.242973 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 982/1985] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:27 2022] PUT /identity/v3/projects/6365712030d540bca7469ed29041ee74/users/24b5ec62da9645a9bf28a7f6e38b2cf2/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:27.255468 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7528f7-43da-4f78-9366-b07feeffe7f8 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:27.255762 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7528f7-43da-4f78-9366-b07feeffe7f8 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:27.256054 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7528f7-43da-4f78-9366-b07feeffe7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:27.285890 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-af7528f7-43da-4f78-9366-b07feeffe7f8 None None] MFA Rules not processed for user `24b5ec62da9645a9bf28a7f6e38b2cf2`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:27.287543 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-af7528f7-43da-4f78-9366-b07feeffe7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:27.300108 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b87cc-a5ab-4913-ad17-2ce133124e93 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:27.302657 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-151b87cc-a5ab-4913-ad17-2ce133124e93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:27.349213 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:27.367505 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1004/1986] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:27.377886 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b87cc-a5ab-4913-ad17-2ce133124e93 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:27.379010 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7528f7-43da-4f78-9366-b07feeffe7f8 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:27.380370 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-af7528f7-43da-4f78-9366-b07feeffe7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:27.382578 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b73f6-7489-47a2-8235-6eb83c148519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:27.383233 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b73f6-7489-47a2-8235-6eb83c148519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:27.383522 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b73f6-7489-47a2-8235-6eb83c148519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:27.384996 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b73f6-7489-47a2-8235-6eb83c148519 admin admin] PATH_INFO: `/v3/projects/92904a77519d4b2f8c5566ec4a2d17c5` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:27.387014 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b73f6-7489-47a2-8235-6eb83c148519 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92904a77519d4b2f8c5566ec4a2d17c5)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:27.388938 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b73f6-7489-47a2-8235-6eb83c148519 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:27.436077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-017b73f6-7489-47a2-8235-6eb83c148519 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92904a77519d4b2f8c5566ec4a2d17c5', 'request_id': 'req-017b73f6-7489-47a2-8235-6eb83c148519'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:27.442387 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-017b73f6-7489-47a2-8235-6eb83c148519 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92904a77519d4b2f8c5566ec4a2d17c5', 'request_id': 'req-017b73f6-7489-47a2-8235-6eb83c148519'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:27.449345 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-017b73f6-7489-47a2-8235-6eb83c148519 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92904a77519d4b2f8c5566ec4a2d17c5', 'request_id': 'req-017b73f6-7489-47a2-8235-6eb83c148519'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:27.464279 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7528f7-43da-4f78-9366-b07feeffe7f8 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:27.468504 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-017b73f6-7489-47a2-8235-6eb83c148519 admin admin] The token cache is being invalidate because project 92904a77519d4b2f8c5566ec4a2d17c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:27.468896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-017b73f6-7489-47a2-8235-6eb83c148519 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:27.469626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-017b73f6-7489-47a2-8235-6eb83c148519 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:27.474475 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 983/1987] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:27 2022] DELETE /identity/v3/projects/92904a77519d4b2f8c5566ec4a2d17c5 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:27.475944 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921e676-8bbf-4d88-a8f9-83c74623cacc None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:27.476171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921e676-8bbf-4d88-a8f9-83c74623cacc None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:27.476450 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921e676-8bbf-4d88-a8f9-83c74623cacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:27.498261 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-8921e676-8bbf-4d88-a8f9-83c74623cacc None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:27.499468 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-8921e676-8bbf-4d88-a8f9-83c74623cacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:27.520007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda16118-a5f9-4268-93e3-c107414c7291 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e98a41a0f304fc2bb36a18a77a2d729', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:27.521092 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda16118-a5f9-4268-93e3-c107414c7291 service glance] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:27.521375 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda16118-a5f9-4268-93e3-c107414c7291 service glance] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:27.521647 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda16118-a5f9-4268-93e3-c107414c7291 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:27.524178 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-fda16118-a5f9-4268-93e3-c107414c7291 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:27.547395 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:27.566991 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 984/1988] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:22:27.576902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f16a3-5160-4f16-8a37-1af83d414c33 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:27.577302 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f16a3-5160-4f16-8a37-1af83d414c33 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:27.577593 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f16a3-5160-4f16-8a37-1af83d414c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:27.592450 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda16118-a5f9-4268-93e3-c107414c7291 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:27.593897 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda16118-a5f9-4268-93e3-c107414c7291 service glance] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:27.599501 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-c58f16a3-5160-4f16-8a37-1af83d414c33 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:27.601098 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c58f16a3-5160-4f16-8a37-1af83d414c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:27.604416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1005/1989] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:27.609688 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 985/1990] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:22:27.614693 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda16118-a5f9-4268-93e3-c107414c7291 service glance] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:27.638444 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda16118-a5f9-4268-93e3-c107414c7291 service glance] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:27.643223 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0155fc9-5ab3-453a-8569-beac810b80c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:27.644303 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0155fc9-5ab3-453a-8569-beac810b80c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:27.644303 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0155fc9-5ab3-453a-8569-beac810b80c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:27.644505 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0155fc9-5ab3-453a-8569-beac810b80c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:27.645073 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0155fc9-5ab3-453a-8569-beac810b80c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:22:27.645374 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0155fc9-5ab3-453a-8569-beac810b80c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:27.646985 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0155fc9-5ab3-453a-8569-beac810b80c8 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:27.662784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1006/1991] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:22:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:22:27.663985 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f16a3-5160-4f16-8a37-1af83d414c33 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:27.673921 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f16a3-5160-4f16-8a37-1af83d414c33 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:27.678685 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556867bb-c362-4801-a11a-6433fc06ae79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:27.679325 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556867bb-c362-4801-a11a-6433fc06ae79 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:27.679612 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556867bb-c362-4801-a11a-6433fc06ae79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:27.679924 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556867bb-c362-4801-a11a-6433fc06ae79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:27.680596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556867bb-c362-4801-a11a-6433fc06ae79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:27.682813 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556867bb-c362-4801-a11a-6433fc06ae79 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:27.715036 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 986/1992] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:27 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:27.719066 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0155fc9-5ab3-453a-8569-beac810b80c8 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:27.729783 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0155fc9-5ab3-453a-8569-beac810b80c8 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:27.789082 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c7b773-f99f-4c27-ac4a-17f9a4cbf3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:27.789662 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7b773-f99f-4c27-ac4a-17f9a4cbf3fb admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:27.789938 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7b773-f99f-4c27-ac4a-17f9a4cbf3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:27.790277 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7b773-f99f-4c27-ac4a-17f9a4cbf3fb admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:27.791649 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c7b773-f99f-4c27-ac4a-17f9a4cbf3fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:27.793552 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c7b773-f99f-4c27-ac4a-17f9a4cbf3fb admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:27.816005 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1007/1993] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:27 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:22:27.821077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556867bb-c362-4801-a11a-6433fc06ae79 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:27.832011 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556867bb-c362-4801-a11a-6433fc06ae79 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:27.838379 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311db59-1e67-4452-8f93-948c35c50dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:27.839017 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311db59-1e67-4452-8f93-948c35c50dc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:27.839300 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311db59-1e67-4452-8f93-948c35c50dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:27.839575 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311db59-1e67-4452-8f93-948c35c50dc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:27.840405 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311db59-1e67-4452-8f93-948c35c50dc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:27.842042 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311db59-1e67-4452-8f93-948c35c50dc2 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:27.852849 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 987/1994] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:27.857441 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c7b773-f99f-4c27-ac4a-17f9a4cbf3fb admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:27.867910 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c7b773-f99f-4c27-ac4a-17f9a4cbf3fb admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:27.873109 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4719f26c-ecb2-4488-ac85-0bfa1c941fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:27.873802 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719f26c-ecb2-4488-ac85-0bfa1c941fba admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:27.874182 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719f26c-ecb2-4488-ac85-0bfa1c941fba admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:27.874527 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719f26c-ecb2-4488-ac85-0bfa1c941fba admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:27.875301 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4719f26c-ecb2-4488-ac85-0bfa1c941fba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:27.876949 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4719f26c-ecb2-4488-ac85-0bfa1c941fba admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:27.889336 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1008/1995] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:27.894426 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311db59-1e67-4452-8f93-948c35c50dc2 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:27.906315 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311db59-1e67-4452-8f93-948c35c50dc2 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:27.911688 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27320ed-2ac2-4e23-bf74-6f152dd03087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:27.912514 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27320ed-2ac2-4e23-bf74-6f152dd03087 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:27.912958 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27320ed-2ac2-4e23-bf74-6f152dd03087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:27.913362 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27320ed-2ac2-4e23-bf74-6f152dd03087 admin admin] PATH_INFO: `/v3/projects/b9a791ac5a2448608226f9817d0912b4/users/05d5503b3eb44f6a91982dd6d2b11e3b/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:27.931241 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27320ed-2ac2-4e23-bf74-6f152dd03087 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9a791ac5a2448608226f9817d0912b4, user_id=05d5503b3eb44f6a91982dd6d2b11e3b, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:27.934177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27320ed-2ac2-4e23-bf74-6f152dd03087 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:27.948564 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 988/1996] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:27 2022] PUT /identity/v3/projects/b9a791ac5a2448608226f9817d0912b4/users/05d5503b3eb44f6a91982dd6d2b11e3b/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:22:27.958939 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b67a94-ed52-4214-84f2-9eb6c163abde None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:27.959230 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b67a94-ed52-4214-84f2-9eb6c163abde None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:27.959535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b67a94-ed52-4214-84f2-9eb6c163abde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:27.988605 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-52b67a94-ed52-4214-84f2-9eb6c163abde None None] MFA Rules not processed for user `05d5503b3eb44f6a91982dd6d2b11e3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:27.989842 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-52b67a94-ed52-4214-84f2-9eb6c163abde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:28.067607 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1009/1997] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:28.072780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27320ed-2ac2-4e23-bf74-6f152dd03087 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:28.078096 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:28.206951 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27320ed-2ac2-4e23-bf74-6f152dd03087 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:28.213066 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831f7a1-da66-4738-9f07-126ea5deef6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:28.213850 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831f7a1-da66-4738-9f07-126ea5deef6f admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:28.214151 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831f7a1-da66-4738-9f07-126ea5deef6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:28.214494 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831f7a1-da66-4738-9f07-126ea5deef6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:28.215289 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831f7a1-da66-4738-9f07-126ea5deef6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:28.217604 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831f7a1-da66-4738-9f07-126ea5deef6f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:28.253991 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 989/1998] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:28 2022] POST /identity/v3/projects => generated 387 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:22:28.261722 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b67a94-ed52-4214-84f2-9eb6c163abde None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:28.284199 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b67a94-ed52-4214-84f2-9eb6c163abde None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:28.298276 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:28.416549 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa05606-60ba-493a-9aca-5621cfbd1244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:28.416863 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa05606-60ba-493a-9aca-5621cfbd1244 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:28.417091 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa05606-60ba-493a-9aca-5621cfbd1244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:28.417500 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa05606-60ba-493a-9aca-5621cfbd1244 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:28.418570 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa05606-60ba-493a-9aca-5621cfbd1244 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:28.421021 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa05606-60ba-493a-9aca-5621cfbd1244 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:28.466435 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1010/1999] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:28 2022] POST /identity/v3/users => generated 300 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:28.469032 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831f7a1-da66-4738-9f07-126ea5deef6f admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:28.480498 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831f7a1-da66-4738-9f07-126ea5deef6f admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:28.486594 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45afdf0c-5386-4039-a93e-1ef238a659c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:28.487639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45afdf0c-5386-4039-a93e-1ef238a659c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:28.487927 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45afdf0c-5386-4039-a93e-1ef238a659c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:28.488205 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45afdf0c-5386-4039-a93e-1ef238a659c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:28.488965 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45afdf0c-5386-4039-a93e-1ef238a659c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:28.490770 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45afdf0c-5386-4039-a93e-1ef238a659c0 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:28.518268 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 990/2000] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:28 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:28.523007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa05606-60ba-493a-9aca-5621cfbd1244 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:28.534176 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa05606-60ba-493a-9aca-5621cfbd1244 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:28.539023 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a3a4a8-bda4-45d9-a726-675f0a751bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:28.539861 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3a4a8-bda4-45d9-a726-675f0a751bda admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:28.540206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3a4a8-bda4-45d9-a726-675f0a751bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:28.540563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3a4a8-bda4-45d9-a726-675f0a751bda admin admin] PATH_INFO: `/v3/projects/447755e239b74dbf8b555cfdd59dbf07/users/d4f77d3dab0243e097b43d66004b3ed0/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:28.557740 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a3a4a8-bda4-45d9-a726-675f0a751bda admin admin] RBAC: Authorizing `identity:create_grant(project_id=447755e239b74dbf8b555cfdd59dbf07, user_id=d4f77d3dab0243e097b43d66004b3ed0, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:28.561193 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a3a4a8-bda4-45d9-a726-675f0a751bda admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:28.577309 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1011/2001] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:28 2022] PUT /identity/v3/projects/447755e239b74dbf8b555cfdd59dbf07/users/d4f77d3dab0243e097b43d66004b3ed0/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:28.581623 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45afdf0c-5386-4039-a93e-1ef238a659c0 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:28.592797 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45afdf0c-5386-4039-a93e-1ef238a659c0 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:28.662186 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f045f86-0f3a-4449-9fe3-12a45394c607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:28.662820 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f045f86-0f3a-4449-9fe3-12a45394c607 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:28.663108 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f045f86-0f3a-4449-9fe3-12a45394c607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:28.663385 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f045f86-0f3a-4449-9fe3-12a45394c607 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:28.664091 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f045f86-0f3a-4449-9fe3-12a45394c607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:28.665737 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f045f86-0f3a-4449-9fe3-12a45394c607 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:28.680023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 991/2002] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:28 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:28.681765 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a3a4a8-bda4-45d9-a726-675f0a751bda admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:28.692791 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a3a4a8-bda4-45d9-a726-675f0a751bda admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:28.698778 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c6753-091e-4e62-9eda-1ca8a6bf0f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:28.699466 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c6753-091e-4e62-9eda-1ca8a6bf0f58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:28.699752 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c6753-091e-4e62-9eda-1ca8a6bf0f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:28.700688 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c6753-091e-4e62-9eda-1ca8a6bf0f58 admin admin] PATH_INFO: `/v3/projects/447755e239b74dbf8b555cfdd59dbf07/users/d4f77d3dab0243e097b43d66004b3ed0/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:28.706667 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62c6753-091e-4e62-9eda-1ca8a6bf0f58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=447755e239b74dbf8b555cfdd59dbf07, user_id=d4f77d3dab0243e097b43d66004b3ed0, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:28.710063 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62c6753-091e-4e62-9eda-1ca8a6bf0f58 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:28.721791 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1012/2003] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:28 2022] PUT /identity/v3/projects/447755e239b74dbf8b555cfdd59dbf07/users/d4f77d3dab0243e097b43d66004b3ed0/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:22:28.733539 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18783f04-5a3f-4e2d-9b26-ee0e1712825e None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:28.733745 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18783f04-5a3f-4e2d-9b26-ee0e1712825e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:28.733967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18783f04-5a3f-4e2d-9b26-ee0e1712825e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:28.777469 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-18783f04-5a3f-4e2d-9b26-ee0e1712825e None None] MFA Rules not processed for user `d4f77d3dab0243e097b43d66004b3ed0`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:28.778661 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-18783f04-5a3f-4e2d-9b26-ee0e1712825e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:28.859590 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:28.865462 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 992/2004] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:28 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:28.877530 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c6753-091e-4e62-9eda-1ca8a6bf0f58 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:28.879529 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b62c6753-091e-4e62-9eda-1ca8a6bf0f58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:28.952840 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c6753-091e-4e62-9eda-1ca8a6bf0f58 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:28.956574 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388fe542-752e-4734-951b-eed57458e3f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:28.957214 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388fe542-752e-4734-951b-eed57458e3f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:28.958017 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388fe542-752e-4734-951b-eed57458e3f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:28.958017 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388fe542-752e-4734-951b-eed57458e3f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:28.978521 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388fe542-752e-4734-951b-eed57458e3f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:28.978521 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388fe542-752e-4734-951b-eed57458e3f4 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:28.981926 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1013/2005] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:28 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:29.012237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18783f04-5a3f-4e2d-9b26-ee0e1712825e None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:29.093083 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18783f04-5a3f-4e2d-9b26-ee0e1712825e None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:29.098797 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65e77d-1e17-46c3-bc46-e5f4f9d8e00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:29.099649 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65e77d-1e17-46c3-bc46-e5f4f9d8e00f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:29.100042 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65e77d-1e17-46c3-bc46-e5f4f9d8e00f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:29.100330 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65e77d-1e17-46c3-bc46-e5f4f9d8e00f admin admin] PATH_INFO: `/v3/users/05d5503b3eb44f6a91982dd6d2b11e3b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:29.102177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd65e77d-1e17-46c3-bc46-e5f4f9d8e00f admin admin] RBAC: Authorizing `identity:delete_user(user_id=05d5503b3eb44f6a91982dd6d2b11e3b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:29.106258 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd65e77d-1e17-46c3-bc46-e5f4f9d8e00f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:29.236444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-fd65e77d-1e17-46c3-bc46-e5f4f9d8e00f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05d5503b3eb44f6a91982dd6d2b11e3b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:29.238430 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-fd65e77d-1e17-46c3-bc46-e5f4f9d8e00f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05d5503b3eb44f6a91982dd6d2b11e3b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:29.250938 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-fd65e77d-1e17-46c3-bc46-e5f4f9d8e00f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05d5503b3eb44f6a91982dd6d2b11e3b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:29.272136 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-fd65e77d-1e17-46c3-bc46-e5f4f9d8e00f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05d5503b3eb44f6a91982dd6d2b11e3b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:29.280866 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-fd65e77d-1e17-46c3-bc46-e5f4f9d8e00f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05d5503b3eb44f6a91982dd6d2b11e3b'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:29.288511 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 993/2006] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:29 2022] DELETE /identity/v3/users/05d5503b3eb44f6a91982dd6d2b11e3b => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:29.297753 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9dfa47-2919-4856-afa6-72e972260bdc None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:29.297753 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9dfa47-2919-4856-afa6-72e972260bdc None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:29.297753 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9dfa47-2919-4856-afa6-72e972260bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:29.328866 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-5c9dfa47-2919-4856-afa6-72e972260bdc None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:29.331599 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5c9dfa47-2919-4856-afa6-72e972260bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:29.419544 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1014/2007] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:22:29.430638 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65e77d-1e17-46c3-bc46-e5f4f9d8e00f admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:29.432972 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-fd65e77d-1e17-46c3-bc46-e5f4f9d8e00f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:29.539189 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65e77d-1e17-46c3-bc46-e5f4f9d8e00f admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:29.544544 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b192838-af88-4018-9bbf-73e716e0ec04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:29.545315 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b192838-af88-4018-9bbf-73e716e0ec04 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:29.545631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b192838-af88-4018-9bbf-73e716e0ec04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:29.546840 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b192838-af88-4018-9bbf-73e716e0ec04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:29.560042 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b192838-af88-4018-9bbf-73e716e0ec04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:29.561641 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b192838-af88-4018-9bbf-73e716e0ec04 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:29.583037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 994/2008] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:29.966477 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9dfa47-2919-4856-afa6-72e972260bdc None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:29.968591 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5c9dfa47-2919-4856-afa6-72e972260bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:29.989755 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9dfa47-2919-4856-afa6-72e972260bdc None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:30.001917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef57a17-49b9-4c08-ba64-630e02f16489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:30.002796 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef57a17-49b9-4c08-ba64-630e02f16489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:30.003227 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef57a17-49b9-4c08-ba64-630e02f16489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:30.003690 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef57a17-49b9-4c08-ba64-630e02f16489 admin admin] PATH_INFO: `/v3/projects/b9a791ac5a2448608226f9817d0912b4` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:30.005511 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef57a17-49b9-4c08-ba64-630e02f16489 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9a791ac5a2448608226f9817d0912b4)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:30.008099 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef57a17-49b9-4c08-ba64-630e02f16489 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:30.012647 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:30.082325 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9ef57a17-49b9-4c08-ba64-630e02f16489 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9a791ac5a2448608226f9817d0912b4', 'request_id': 'req-9ef57a17-49b9-4c08-ba64-630e02f16489'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:30.093060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9ef57a17-49b9-4c08-ba64-630e02f16489 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9a791ac5a2448608226f9817d0912b4', 'request_id': 'req-9ef57a17-49b9-4c08-ba64-630e02f16489'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:30.108077 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9ef57a17-49b9-4c08-ba64-630e02f16489 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9a791ac5a2448608226f9817d0912b4', 'request_id': 'req-9ef57a17-49b9-4c08-ba64-630e02f16489'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:30.122047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9ef57a17-49b9-4c08-ba64-630e02f16489 admin admin] The token cache is being invalidate because project b9a791ac5a2448608226f9817d0912b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:30.122245 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9ef57a17-49b9-4c08-ba64-630e02f16489 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:30.122909 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9ef57a17-49b9-4c08-ba64-630e02f16489 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:30.124592 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1015/2009] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:29 2022] DELETE /identity/v3/projects/b9a791ac5a2448608226f9817d0912b4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:30.128475 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b192838-af88-4018-9bbf-73e716e0ec04 service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:30.140828 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b192838-af88-4018-9bbf-73e716e0ec04 service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:30.163023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:30.214387 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d80e77-c4f9-4de5-80ce-3af318e0a539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:30.215028 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d80e77-c4f9-4de5-80ce-3af318e0a539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:30.215312 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d80e77-c4f9-4de5-80ce-3af318e0a539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:30.215588 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d80e77-c4f9-4de5-80ce-3af318e0a539 admin admin] PATH_INFO: `/v3/users/d4f77d3dab0243e097b43d66004b3ed0` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:30.216753 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d80e77-c4f9-4de5-80ce-3af318e0a539 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4f77d3dab0243e097b43d66004b3ed0)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:30.218732 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d80e77-c4f9-4de5-80ce-3af318e0a539 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:30.276553 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-70d80e77-c4f9-4de5-80ce-3af318e0a539 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4f77d3dab0243e097b43d66004b3ed0'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:30.277326 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-70d80e77-c4f9-4de5-80ce-3af318e0a539 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4f77d3dab0243e097b43d66004b3ed0'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:30.283947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-70d80e77-c4f9-4de5-80ce-3af318e0a539 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4f77d3dab0243e097b43d66004b3ed0'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:30.298077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-70d80e77-c4f9-4de5-80ce-3af318e0a539 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4f77d3dab0243e097b43d66004b3ed0'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:30.304870 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-70d80e77-c4f9-4de5-80ce-3af318e0a539 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4f77d3dab0243e097b43d66004b3ed0'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:30.311856 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 995/2010] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:30 2022] DELETE /identity/v3/users/d4f77d3dab0243e097b43d66004b3ed0 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:30.510816 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef57a17-49b9-4c08-ba64-630e02f16489 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:30.512828 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-9ef57a17-49b9-4c08-ba64-630e02f16489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:30.537817 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef57a17-49b9-4c08-ba64-630e02f16489 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:30.629867 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb9e24-5467-4c85-974d-2d80bcfe0d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:30.630893 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb9e24-5467-4c85-974d-2d80bcfe0d39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:30.631413 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb9e24-5467-4c85-974d-2d80bcfe0d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:30.631838 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb9e24-5467-4c85-974d-2d80bcfe0d39 admin admin] PATH_INFO: `/v3/users/5663b9134b7544e088ddebf242f8db7e` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:30.633250 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb9e24-5467-4c85-974d-2d80bcfe0d39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5663b9134b7544e088ddebf242f8db7e)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:30.636440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb9e24-5467-4c85-974d-2d80bcfe0d39 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:30.689360 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf601e0-f4c4-4cea-a3a5-c9bbc5815770 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:30.689689 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf601e0-f4c4-4cea-a3a5-c9bbc5815770 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:30.689924 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf601e0-f4c4-4cea-a3a5-c9bbc5815770 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:30.697778 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 996/2011] 10.222.0.46 () {54 vars in 921 bytes} [Wed Jul 27 04:22:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:22:30.707164 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b40ceb-c91f-434c-8c33-d84da5f65682 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:30.707457 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b40ceb-c91f-434c-8c33-d84da5f65682 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:30.707960 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b40ceb-c91f-434c-8c33-d84da5f65682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:30.747575 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-b4b40ceb-c91f-434c-8c33-d84da5f65682 None None] MFA Rules not processed for user `9dc9ea8f89c14a2a8e3272f750647dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:30.749231 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b4b40ceb-c91f-434c-8c33-d84da5f65682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:30.754131 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-78cb9e24-5467-4c85-974d-2d80bcfe0d39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5663b9134b7544e088ddebf242f8db7e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:30.754843 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-78cb9e24-5467-4c85-974d-2d80bcfe0d39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5663b9134b7544e088ddebf242f8db7e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:30.757246 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-78cb9e24-5467-4c85-974d-2d80bcfe0d39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5663b9134b7544e088ddebf242f8db7e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:30.778586 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-78cb9e24-5467-4c85-974d-2d80bcfe0d39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5663b9134b7544e088ddebf242f8db7e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:30.797086 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-78cb9e24-5467-4c85-974d-2d80bcfe0d39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5663b9134b7544e088ddebf242f8db7e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:30.832937 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1016/2012] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:30 2022] DELETE /identity/v3/users/5663b9134b7544e088ddebf242f8db7e => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:30.833810 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb9e24-5467-4c85-974d-2d80bcfe0d39 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:30.836864 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-78cb9e24-5467-4c85-974d-2d80bcfe0d39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:30.855358 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:30.878244 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 997/2013] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Jul 27 04:22:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:30.882019 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a88fb-da58-458f-b35e-51d4e8f4d4fb None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:30.882019 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a88fb-da58-458f-b35e-51d4e8f4d4fb None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:30.882019 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a88fb-da58-458f-b35e-51d4e8f4d4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:30.933368 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-fd1a88fb-da58-458f-b35e-51d4e8f4d4fb None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:30.936910 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-fd1a88fb-da58-458f-b35e-51d4e8f4d4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:30.955326 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb9e24-5467-4c85-974d-2d80bcfe0d39 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:30.961237 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a377818-d7a0-454b-8fe8-bf55cda9955c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:30.961952 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a377818-d7a0-454b-8fe8-bf55cda9955c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:30.963164 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a377818-d7a0-454b-8fe8-bf55cda9955c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:30.963629 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a377818-d7a0-454b-8fe8-bf55cda9955c admin admin] PATH_INFO: `/v3/projects/447755e239b74dbf8b555cfdd59dbf07` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:30.964880 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a377818-d7a0-454b-8fe8-bf55cda9955c admin admin] RBAC: Authorizing `identity:delete_project(project_id=447755e239b74dbf8b555cfdd59dbf07)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:30.967140 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a377818-d7a0-454b-8fe8-bf55cda9955c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:30.969429 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 998/2014] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:30.970326 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a88fb-da58-458f-b35e-51d4e8f4d4fb None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:30.985180 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-fd1a88fb-da58-458f-b35e-51d4e8f4d4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:31.023699 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9a377818-d7a0-454b-8fe8-bf55cda9955c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '447755e239b74dbf8b555cfdd59dbf07', 'request_id': 'req-9a377818-d7a0-454b-8fe8-bf55cda9955c'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:31.035263 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9a377818-d7a0-454b-8fe8-bf55cda9955c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '447755e239b74dbf8b555cfdd59dbf07', 'request_id': 'req-9a377818-d7a0-454b-8fe8-bf55cda9955c'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:31.044354 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9a377818-d7a0-454b-8fe8-bf55cda9955c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '447755e239b74dbf8b555cfdd59dbf07', 'request_id': 'req-9a377818-d7a0-454b-8fe8-bf55cda9955c'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:31.048141 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:31.072162 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9a377818-d7a0-454b-8fe8-bf55cda9955c admin admin] The token cache is being invalidate because project 447755e239b74dbf8b555cfdd59dbf07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:31.072527 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9a377818-d7a0-454b-8fe8-bf55cda9955c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:31.074600 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9a377818-d7a0-454b-8fe8-bf55cda9955c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:31.076269 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1017/2015] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:30 2022] DELETE /identity/v3/projects/447755e239b74dbf8b555cfdd59dbf07 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:31.086788 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a377818-d7a0-454b-8fe8-bf55cda9955c admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:31.090509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-9a377818-d7a0-454b-8fe8-bf55cda9955c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:31.152336 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a88fb-da58-458f-b35e-51d4e8f4d4fb None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:31.298246 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a377818-d7a0-454b-8fe8-bf55cda9955c admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:31.308333 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e202ca-5257-4617-8c49-f62ccc3a6164 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:31.309012 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e202ca-5257-4617-8c49-f62ccc3a6164 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:31.309317 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e202ca-5257-4617-8c49-f62ccc3a6164 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:31.311528 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e202ca-5257-4617-8c49-f62ccc3a6164 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:31.314773 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-56e202ca-5257-4617-8c49-f62ccc3a6164 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:31.350367 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f5145b-ba0e-4988-88dc-e0ba97b74d37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:31.353636 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5145b-ba0e-4988-88dc-e0ba97b74d37 service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:31.354348 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5145b-ba0e-4988-88dc-e0ba97b74d37 service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:31.356503 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5145b-ba0e-4988-88dc-e0ba97b74d37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:31.363725 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-31f5145b-ba0e-4988-88dc-e0ba97b74d37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:31.387155 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f5145b-ba0e-4988-88dc-e0ba97b74d37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:31.390416 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f5145b-ba0e-4988-88dc-e0ba97b74d37 service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:31.417412 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 999/2016] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:31.420213 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f1341-889e-470a-b842-a69c68d8b54f None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:31.420485 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f1341-889e-470a-b842-a69c68d8b54f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:31.423631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f1341-889e-470a-b842-a69c68d8b54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:31.425555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:31.447189 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e202ca-5257-4617-8c49-f62ccc3a6164 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:31.453844 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e202ca-5257-4617-8c49-f62ccc3a6164 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:31.468665 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-359f1341-889e-470a-b842-a69c68d8b54f None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:31.471528 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-359f1341-889e-470a-b842-a69c68d8b54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:31.478886 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1018/2017] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:31.490021 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1000/2018] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:31.497559 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ce60b-9b6e-474e-a59d-f329884b3783 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:31.498060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ce60b-9b6e-474e-a59d-f329884b3783 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:31.499377 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ce60b-9b6e-474e-a59d-f329884b3783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:31.529404 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-f01ce60b-9b6e-474e-a59d-f329884b3783 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:31.531236 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f01ce60b-9b6e-474e-a59d-f329884b3783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:31.550835 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1019/2019] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:31.555681 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f1341-889e-470a-b842-a69c68d8b54f None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:31.569189 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f1341-889e-470a-b842-a69c68d8b54f None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:31.581131 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40edc1e7-e120-48cf-9318-7d8a40c3fb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:31.581693 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40edc1e7-e120-48cf-9318-7d8a40c3fb7a admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:31.582023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40edc1e7-e120-48cf-9318-7d8a40c3fb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:31.583055 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40edc1e7-e120-48cf-9318-7d8a40c3fb7a admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:31.583780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40edc1e7-e120-48cf-9318-7d8a40c3fb7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:22:31.586167 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40edc1e7-e120-48cf-9318-7d8a40c3fb7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:31.588968 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40edc1e7-e120-48cf-9318-7d8a40c3fb7a admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:31.611101 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1001/2020] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:22:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:22:31.623928 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ce60b-9b6e-474e-a59d-f329884b3783 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:31.638741 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ce60b-9b6e-474e-a59d-f329884b3783 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:31.646447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aebc217-d54f-49cf-ae74-d822a8149ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:31.647377 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aebc217-d54f-49cf-ae74-d822a8149ace admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:31.647910 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aebc217-d54f-49cf-ae74-d822a8149ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:31.648402 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aebc217-d54f-49cf-ae74-d822a8149ace admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:31.649203 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aebc217-d54f-49cf-ae74-d822a8149ace admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:31.655996 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aebc217-d54f-49cf-ae74-d822a8149ace admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:31.719228 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1020/2021] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:31 2022] POST /identity/v3/projects => generated 381 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:31.723369 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40edc1e7-e120-48cf-9318-7d8a40c3fb7a admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:31.735657 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40edc1e7-e120-48cf-9318-7d8a40c3fb7a admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:31.848737 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ee8fc-c33d-4a1f-88d9-b5a542a232d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:31.849525 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ee8fc-c33d-4a1f-88d9-b5a542a232d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:31.849781 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ee8fc-c33d-4a1f-88d9-b5a542a232d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:31.850094 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ee8fc-c33d-4a1f-88d9-b5a542a232d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:31.850793 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934ee8fc-c33d-4a1f-88d9-b5a542a232d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:31.854635 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934ee8fc-c33d-4a1f-88d9-b5a542a232d1 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:31.867446 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aebc217-d54f-49cf-ae74-d822a8149ace admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:31.869804 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-1aebc217-d54f-49cf-ae74-d822a8149ace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:31.879680 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1002/2022] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:31 2022] POST /identity/v3/users => generated 291 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:31.884815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ee8fc-c33d-4a1f-88d9-b5a542a232d1 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:31.887387 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aebc217-d54f-49cf-ae74-d822a8149ace admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:31.893867 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8551539f-5667-4aff-b96a-4160e75b0db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:31.894598 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8551539f-5667-4aff-b96a-4160e75b0db5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:31.894892 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8551539f-5667-4aff-b96a-4160e75b0db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:31.895168 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8551539f-5667-4aff-b96a-4160e75b0db5 admin admin] PATH_INFO: `/v3/users/93c2e138a825454999ae5caf8e164e87` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:31.896451 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ee8fc-c33d-4a1f-88d9-b5a542a232d1 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:31.896573 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8551539f-5667-4aff-b96a-4160e75b0db5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93c2e138a825454999ae5caf8e164e87)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:31.898442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8551539f-5667-4aff-b96a-4160e75b0db5 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:31.902927 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38de916-016f-4e8a-af35-849568a37bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:31.903553 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38de916-016f-4e8a-af35-849568a37bfe admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:31.903835 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38de916-016f-4e8a-af35-849568a37bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:31.904111 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38de916-016f-4e8a-af35-849568a37bfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:31.904968 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38de916-016f-4e8a-af35-849568a37bfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:31.906940 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38de916-016f-4e8a-af35-849568a37bfe admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:31.923120 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1003/2023] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:31.927117 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38de916-016f-4e8a-af35-849568a37bfe admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:31.940244 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38de916-016f-4e8a-af35-849568a37bfe admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:31.946295 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a61e08-2274-46d7-b5da-b7c6eb00c22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:31.947462 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a61e08-2274-46d7-b5da-b7c6eb00c22b admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:31.949356 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a61e08-2274-46d7-b5da-b7c6eb00c22b admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:31.950025 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a61e08-2274-46d7-b5da-b7c6eb00c22b admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:31.950464 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a61e08-2274-46d7-b5da-b7c6eb00c22b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:31.952098 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a61e08-2274-46d7-b5da-b7c6eb00c22b admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:31.955179 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:31.964807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8551539f-5667-4aff-b96a-4160e75b0db5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93c2e138a825454999ae5caf8e164e87'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:31.969181 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1004/2024] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:31.971648 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a61e08-2274-46d7-b5da-b7c6eb00c22b admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:31.973227 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8551539f-5667-4aff-b96a-4160e75b0db5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93c2e138a825454999ae5caf8e164e87'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:31.974458 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8551539f-5667-4aff-b96a-4160e75b0db5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93c2e138a825454999ae5caf8e164e87'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:31.984635 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a61e08-2274-46d7-b5da-b7c6eb00c22b admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:31.992000 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea12cf-5d9f-4566-bfa9-dc0dbd771a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:31.992675 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8551539f-5667-4aff-b96a-4160e75b0db5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93c2e138a825454999ae5caf8e164e87'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:31.994380 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea12cf-5d9f-4566-bfa9-dc0dbd771a77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:31.994909 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea12cf-5d9f-4566-bfa9-dc0dbd771a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:31.995477 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea12cf-5d9f-4566-bfa9-dc0dbd771a77 admin admin] PATH_INFO: `/v3/projects/192024afe10f477bb8cecd9fe1f44d14/users/ec2fda6d3b744369a27bbb57a7b61232/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:32.005468 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8551539f-5667-4aff-b96a-4160e75b0db5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93c2e138a825454999ae5caf8e164e87'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:32.019208 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ea12cf-5d9f-4566-bfa9-dc0dbd771a77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=192024afe10f477bb8cecd9fe1f44d14, user_id=ec2fda6d3b744369a27bbb57a7b61232, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:32.023301 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ea12cf-5d9f-4566-bfa9-dc0dbd771a77 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:32.026283 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1021/2025] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:31 2022] DELETE /identity/v3/users/93c2e138a825454999ae5caf8e164e87 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:32.031618 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f793d06-6cc7-48b6-836e-ca0aeb6d6021 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:32.032011 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f793d06-6cc7-48b6-836e-ca0aeb6d6021 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:32.032347 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f793d06-6cc7-48b6-836e-ca0aeb6d6021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:32.046634 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1005/2026] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:31 2022] PUT /identity/v3/projects/192024afe10f477bb8cecd9fe1f44d14/users/ec2fda6d3b744369a27bbb57a7b61232/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:22:32.047616 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea12cf-5d9f-4566-bfa9-dc0dbd771a77 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:32.049530 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-75ea12cf-5d9f-4566-bfa9-dc0dbd771a77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:32.055602 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-2f793d06-6cc7-48b6-836e-ca0aeb6d6021 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:32.057748 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2f793d06-6cc7-48b6-836e-ca0aeb6d6021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:32.135099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea12cf-5d9f-4566-bfa9-dc0dbd771a77 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:32.137724 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1022/2027] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:32.141933 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43c754-5466-4bda-844c-2b3792fcf9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:32.142450 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43c754-5466-4bda-844c-2b3792fcf9d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:32.142739 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43c754-5466-4bda-844c-2b3792fcf9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:32.143014 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43c754-5466-4bda-844c-2b3792fcf9d1 admin admin] PATH_INFO: `/v3/projects/37a09d2936ee4cda9d1fe016e604b174` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:32.143486 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf82d3-ecf0-475a-b98b-957a2ae6f06d None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:32.143877 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf82d3-ecf0-475a-b98b-957a2ae6f06d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:32.144312 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43c754-5466-4bda-844c-2b3792fcf9d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37a09d2936ee4cda9d1fe016e604b174)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:32.144426 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf82d3-ecf0-475a-b98b-957a2ae6f06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:32.146653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43c754-5466-4bda-844c-2b3792fcf9d1 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:32.179776 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-7abf82d3-ecf0-475a-b98b-957a2ae6f06d None None] MFA Rules not processed for user `ec2fda6d3b744369a27bbb57a7b61232`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:32.182549 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7abf82d3-ecf0-475a-b98b-957a2ae6f06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:32.209462 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2b43c754-5466-4bda-844c-2b3792fcf9d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37a09d2936ee4cda9d1fe016e604b174', 'request_id': 'req-2b43c754-5466-4bda-844c-2b3792fcf9d1'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:32.213300 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:32.219716 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2b43c754-5466-4bda-844c-2b3792fcf9d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37a09d2936ee4cda9d1fe016e604b174', 'request_id': 'req-2b43c754-5466-4bda-844c-2b3792fcf9d1'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:32.230896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2b43c754-5466-4bda-844c-2b3792fcf9d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37a09d2936ee4cda9d1fe016e604b174', 'request_id': 'req-2b43c754-5466-4bda-844c-2b3792fcf9d1'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:32.247778 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2b43c754-5466-4bda-844c-2b3792fcf9d1 admin admin] The token cache is being invalidate because project 37a09d2936ee4cda9d1fe016e604b174 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:32.248239 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2b43c754-5466-4bda-844c-2b3792fcf9d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:32.250313 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2b43c754-5466-4bda-844c-2b3792fcf9d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:32.255463 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1006/2028] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:32 2022] DELETE /identity/v3/projects/37a09d2936ee4cda9d1fe016e604b174 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:32.255463 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43c754-5466-4bda-844c-2b3792fcf9d1 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:32.256476 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2b43c754-5466-4bda-844c-2b3792fcf9d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:32.282600 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1023/2029] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:32.283683 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abf82d3-ecf0-475a-b98b-957a2ae6f06d None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:32.288202 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7abf82d3-ecf0-475a-b98b-957a2ae6f06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:32.321242 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abf82d3-ecf0-475a-b98b-957a2ae6f06d None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:32.363643 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43c754-5466-4bda-844c-2b3792fcf9d1 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:32.369800 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a9c8c9-6ff9-4127-b4f0-851915620917 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:32.370483 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9c8c9-6ff9-4127-b4f0-851915620917 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:32.370775 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9c8c9-6ff9-4127-b4f0-851915620917 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:32.371867 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9c8c9-6ff9-4127-b4f0-851915620917 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:32.373191 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-85a9c8c9-6ff9-4127-b4f0-851915620917 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:32.389584 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:32.411233 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6edca-4dac-4b47-95e9-7639209499e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:32.412966 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6edca-4dac-4b47-95e9-7639209499e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:32.413303 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6edca-4dac-4b47-95e9-7639209499e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:32.413583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6edca-4dac-4b47-95e9-7639209499e9 admin admin] PATH_INFO: `/v3/users/ee564943e2da4b0d8134e815a637180a` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:32.414891 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb6edca-4dac-4b47-95e9-7639209499e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee564943e2da4b0d8134e815a637180a)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:32.417144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb6edca-4dac-4b47-95e9-7639209499e9 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:32.471944 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a9c8c9-6ff9-4127-b4f0-851915620917 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:32.474156 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a9c8c9-6ff9-4127-b4f0-851915620917 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:32.507209 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-cfb6edca-4dac-4b47-95e9-7639209499e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee564943e2da4b0d8134e815a637180a'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:32.507791 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1007/2030] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:32.509944 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-cfb6edca-4dac-4b47-95e9-7639209499e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee564943e2da4b0d8134e815a637180a'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:32.511270 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-cfb6edca-4dac-4b47-95e9-7639209499e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee564943e2da4b0d8134e815a637180a'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:32.514733 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a9c8c9-6ff9-4127-b4f0-851915620917 service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:32.519470 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-85a9c8c9-6ff9-4127-b4f0-851915620917 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:32.539876 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a9c8c9-6ff9-4127-b4f0-851915620917 service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:32.554739 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704fe483-e484-41e7-a27c-b77252c6d209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:32.555390 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704fe483-e484-41e7-a27c-b77252c6d209 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:32.555677 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704fe483-e484-41e7-a27c-b77252c6d209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:32.556623 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704fe483-e484-41e7-a27c-b77252c6d209 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:32.556898 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-cfb6edca-4dac-4b47-95e9-7639209499e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee564943e2da4b0d8134e815a637180a'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:32.557347 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704fe483-e484-41e7-a27c-b77252c6d209 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:32.562879 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704fe483-e484-41e7-a27c-b77252c6d209 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:32.578662 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-cfb6edca-4dac-4b47-95e9-7639209499e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee564943e2da4b0d8134e815a637180a'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:32.613515 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1024/2031] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:32 2022] DELETE /identity/v3/users/ee564943e2da4b0d8134e815a637180a => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:32.641738 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1008/2032] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:32 2022] POST /identity/v3/projects => generated 379 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:32.642991 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6edca-4dac-4b47-95e9-7639209499e9 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:32.646243 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-cfb6edca-4dac-4b47-95e9-7639209499e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:32.706513 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:32.732203 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6edca-4dac-4b47-95e9-7639209499e9 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:32.822378 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f7488-2942-4c4b-9fb1-aad83ad3abad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:32.823537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f7488-2942-4c4b-9fb1-aad83ad3abad admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:32.823927 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f7488-2942-4c4b-9fb1-aad83ad3abad admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:32.824434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f7488-2942-4c4b-9fb1-aad83ad3abad admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:32.825335 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10f7488-2942-4c4b-9fb1-aad83ad3abad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:32.827706 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10f7488-2942-4c4b-9fb1-aad83ad3abad admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:32.856826 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1025/2033] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:32 2022] POST /identity/v3/users => generated 296 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:22:32.861521 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704fe483-e484-41e7-a27c-b77252c6d209 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:32.874311 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704fe483-e484-41e7-a27c-b77252c6d209 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:32.879801 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c453c-9d8f-4a2a-ad42-64e597002862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:32.880911 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c453c-9d8f-4a2a-ad42-64e597002862 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:32.881110 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c453c-9d8f-4a2a-ad42-64e597002862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:32.881501 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c453c-9d8f-4a2a-ad42-64e597002862 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:32.882392 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c453c-9d8f-4a2a-ad42-64e597002862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:32.884428 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c453c-9d8f-4a2a-ad42-64e597002862 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:32.901313 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1009/2034] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:32.901952 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f7488-2942-4c4b-9fb1-aad83ad3abad admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:32.924761 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f7488-2942-4c4b-9fb1-aad83ad3abad admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:32.931277 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0e8ea-b51f-4711-91c4-e15cdbe179ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:32.932243 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0e8ea-b51f-4711-91c4-e15cdbe179ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:32.932533 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0e8ea-b51f-4711-91c4-e15cdbe179ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:32.932817 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0e8ea-b51f-4711-91c4-e15cdbe179ba admin admin] PATH_INFO: `/v3/projects/b594579fd4534918acf5fd1e38cff36b/users/b01fc37aab0c407d9ca6aeecff422159/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:32.944349 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c453c-9d8f-4a2a-ad42-64e597002862 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:32.947011 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-dd3c453c-9d8f-4a2a-ad42-64e597002862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:32.954801 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d0e8ea-b51f-4711-91c4-e15cdbe179ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=b594579fd4534918acf5fd1e38cff36b, user_id=b01fc37aab0c407d9ca6aeecff422159, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:32.957894 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d0e8ea-b51f-4711-91c4-e15cdbe179ba admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:32.970276 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c453c-9d8f-4a2a-ad42-64e597002862 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:32.978732 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee74abb9-0bc4-4ac9-9a77-e4b50c44675a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:32.979626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74abb9-0bc4-4ac9-9a77-e4b50c44675a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:32.979918 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74abb9-0bc4-4ac9-9a77-e4b50c44675a admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:32.980372 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74abb9-0bc4-4ac9-9a77-e4b50c44675a admin admin] PATH_INFO: `/v3/projects/5ac6fc9debb64c41a5a943c0991c748d` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:32.982556 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee74abb9-0bc4-4ac9-9a77-e4b50c44675a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ac6fc9debb64c41a5a943c0991c748d)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:32.983999 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1026/2035] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:32 2022] PUT /identity/v3/projects/b594579fd4534918acf5fd1e38cff36b/users/b01fc37aab0c407d9ca6aeecff422159/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:22:32.984672 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee74abb9-0bc4-4ac9-9a77-e4b50c44675a admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:32.988431 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0e8ea-b51f-4711-91c4-e15cdbe179ba admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:33.060583 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ee74abb9-0bc4-4ac9-9a77-e4b50c44675a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ac6fc9debb64c41a5a943c0991c748d', 'request_id': 'req-ee74abb9-0bc4-4ac9-9a77-e4b50c44675a'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:33.067463 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0e8ea-b51f-4711-91c4-e15cdbe179ba admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:33.067840 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ee74abb9-0bc4-4ac9-9a77-e4b50c44675a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ac6fc9debb64c41a5a943c0991c748d', 'request_id': 'req-ee74abb9-0bc4-4ac9-9a77-e4b50c44675a'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:33.075103 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ee74abb9-0bc4-4ac9-9a77-e4b50c44675a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ac6fc9debb64c41a5a943c0991c748d', 'request_id': 'req-ee74abb9-0bc4-4ac9-9a77-e4b50c44675a'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:33.089239 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ee74abb9-0bc4-4ac9-9a77-e4b50c44675a admin admin] The token cache is being invalidate because project 5ac6fc9debb64c41a5a943c0991c748d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:33.089602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ee74abb9-0bc4-4ac9-9a77-e4b50c44675a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:33.090420 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ee74abb9-0bc4-4ac9-9a77-e4b50c44675a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:33.092032 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1010/2036] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:32 2022] DELETE /identity/v3/projects/5ac6fc9debb64c41a5a943c0991c748d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:22:33.096332 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee74abb9-0bc4-4ac9-9a77-e4b50c44675a admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:33.097699 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ee74abb9-0bc4-4ac9-9a77-e4b50c44675a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:33.134726 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89024da-2cf8-499d-8672-c1329ac64864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:33.135291 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89024da-2cf8-499d-8672-c1329ac64864 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:33.135519 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89024da-2cf8-499d-8672-c1329ac64864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:33.135793 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89024da-2cf8-499d-8672-c1329ac64864 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:33.136502 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89024da-2cf8-499d-8672-c1329ac64864 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:33.138161 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89024da-2cf8-499d-8672-c1329ac64864 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:33.143719 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:33.150451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1027/2037] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:32 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:33.151778 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0c93d-4be5-4834-9c92-bb23df8549a5 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:33.152132 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0c93d-4be5-4834-9c92-bb23df8549a5 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:33.152418 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0c93d-4be5-4834-9c92-bb23df8549a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:33.174056 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-82b0c93d-4be5-4834-9c92-bb23df8549a5 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:33.174264 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee74abb9-0bc4-4ac9-9a77-e4b50c44675a admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:33.175299 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-82b0c93d-4be5-4834-9c92-bb23df8549a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:33.179175 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61f5b8-9618-4de6-aa94-524511e68b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:33.179836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61f5b8-9618-4de6-aa94-524511e68b0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:33.180124 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61f5b8-9618-4de6-aa94-524511e68b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:33.180404 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61f5b8-9618-4de6-aa94-524511e68b0d admin admin] PATH_INFO: `/v3/projects/288d611a37a1446dab8e4465d5c90ca6` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:33.182222 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61f5b8-9618-4de6-aa94-524511e68b0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=288d611a37a1446dab8e4465d5c90ca6)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:33.184136 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1028/2038] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:33.184254 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61f5b8-9618-4de6-aa94-524511e68b0d admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:33.185063 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0c93d-4be5-4834-9c92-bb23df8549a5 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:33.186440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-82b0c93d-4be5-4834-9c92-bb23df8549a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:33.191516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:33.200133 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0c93d-4be5-4834-9c92-bb23df8549a5 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:33.205692 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe905d-7465-4503-9ba2-2904666674a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:33.206553 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe905d-7465-4503-9ba2-2904666674a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:33.206840 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe905d-7465-4503-9ba2-2904666674a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:33.207121 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe905d-7465-4503-9ba2-2904666674a3 admin admin] PATH_INFO: `/v3/projects/b594579fd4534918acf5fd1e38cff36b/users/b01fc37aab0c407d9ca6aeecff422159/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:33.209309 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe905d-7465-4503-9ba2-2904666674a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b594579fd4534918acf5fd1e38cff36b, user_id=b01fc37aab0c407d9ca6aeecff422159, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:33.212437 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe905d-7465-4503-9ba2-2904666674a3 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:33.224382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1029/2039] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:33 2022] PUT /identity/v3/projects/b594579fd4534918acf5fd1e38cff36b/users/b01fc37aab0c407d9ca6aeecff422159/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:33.225943 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386f3d5-c80b-4d07-8a30-a24bf9caccb7 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:33.226256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386f3d5-c80b-4d07-8a30-a24bf9caccb7 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:33.226663 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386f3d5-c80b-4d07-8a30-a24bf9caccb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:33.247594 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-9d61f5b8-9618-4de6-aa94-524511e68b0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '288d611a37a1446dab8e4465d5c90ca6', 'request_id': 'req-9d61f5b8-9618-4de6-aa94-524511e68b0d'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:33.249667 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-7386f3d5-c80b-4d07-8a30-a24bf9caccb7 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:33.250943 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7386f3d5-c80b-4d07-8a30-a24bf9caccb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:33.253819 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-9d61f5b8-9618-4de6-aa94-524511e68b0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '288d611a37a1446dab8e4465d5c90ca6', 'request_id': 'req-9d61f5b8-9618-4de6-aa94-524511e68b0d'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:33.261474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-9d61f5b8-9618-4de6-aa94-524511e68b0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '288d611a37a1446dab8e4465d5c90ca6', 'request_id': 'req-9d61f5b8-9618-4de6-aa94-524511e68b0d'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:33.276476 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-9d61f5b8-9618-4de6-aa94-524511e68b0d admin admin] The token cache is being invalidate because project 288d611a37a1446dab8e4465d5c90ca6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:33.276768 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-9d61f5b8-9618-4de6-aa94-524511e68b0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:33.277425 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-9d61f5b8-9618-4de6-aa94-524511e68b0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:33.279195 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1011/2040] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:33 2022] DELETE /identity/v3/projects/288d611a37a1446dab8e4465d5c90ca6 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:33.280226 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aeb26f-4a56-4a1c-9e30-5507de58bac5 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:33.280530 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aeb26f-4a56-4a1c-9e30-5507de58bac5 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:33.280804 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aeb26f-4a56-4a1c-9e30-5507de58bac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:33.311246 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-73aeb26f-4a56-4a1c-9e30-5507de58bac5 None None] MFA Rules not processed for user `b01fc37aab0c407d9ca6aeecff422159`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:33.312470 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-73aeb26f-4a56-4a1c-9e30-5507de58bac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:33.322326 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1030/2041] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:33.323331 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7386f3d5-c80b-4d07-8a30-a24bf9caccb7 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:33.324692 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7386f3d5-c80b-4d07-8a30-a24bf9caccb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:33.336609 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7386f3d5-c80b-4d07-8a30-a24bf9caccb7 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:33.343144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a7f9e-e9aa-4130-9f74-4dd9e746d532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:33.343794 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a7f9e-e9aa-4130-9f74-4dd9e746d532 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:33.344080 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a7f9e-e9aa-4130-9f74-4dd9e746d532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:33.344356 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a7f9e-e9aa-4130-9f74-4dd9e746d532 admin admin] PATH_INFO: `/v3/users/f373ca22369b4b3fb77d7a45b66e9a82` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:33.345593 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a7f9e-e9aa-4130-9f74-4dd9e746d532 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f373ca22369b4b3fb77d7a45b66e9a82)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:33.347889 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a7f9e-e9aa-4130-9f74-4dd9e746d532 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:33.404203 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fe1a7f9e-e9aa-4130-9f74-4dd9e746d532 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f373ca22369b4b3fb77d7a45b66e9a82'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:33.404984 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fe1a7f9e-e9aa-4130-9f74-4dd9e746d532 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f373ca22369b4b3fb77d7a45b66e9a82'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:33.405687 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fe1a7f9e-e9aa-4130-9f74-4dd9e746d532 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f373ca22369b4b3fb77d7a45b66e9a82'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:33.413453 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1012/2042] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:33 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:33.414414 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aeb26f-4a56-4a1c-9e30-5507de58bac5 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:33.425655 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aeb26f-4a56-4a1c-9e30-5507de58bac5 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:33.426599 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fe1a7f9e-e9aa-4130-9f74-4dd9e746d532 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f373ca22369b4b3fb77d7a45b66e9a82'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:33.432700 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2569fcbd-7525-4453-b08b-16b66b16e708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:33.433347 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569fcbd-7525-4453-b08b-16b66b16e708 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:33.434264 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569fcbd-7525-4453-b08b-16b66b16e708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:33.434557 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569fcbd-7525-4453-b08b-16b66b16e708 admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:33.435670 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2569fcbd-7525-4453-b08b-16b66b16e708 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:22:33.436431 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2569fcbd-7525-4453-b08b-16b66b16e708 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:33.436634 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fe1a7f9e-e9aa-4130-9f74-4dd9e746d532 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f373ca22369b4b3fb77d7a45b66e9a82'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:33.437869 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2569fcbd-7525-4453-b08b-16b66b16e708 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:33.457756 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1013/2043] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:22:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:22:33.457756 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2569fcbd-7525-4453-b08b-16b66b16e708 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:33.460220 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2569fcbd-7525-4453-b08b-16b66b16e708 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:33.465882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1031/2044] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:33 2022] DELETE /identity/v3/users/f373ca22369b4b3fb77d7a45b66e9a82 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:33.470884 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a7f9e-e9aa-4130-9f74-4dd9e746d532 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:33.472919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-fe1a7f9e-e9aa-4130-9f74-4dd9e746d532 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:33.553706 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2569fcbd-7525-4453-b08b-16b66b16e708 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:33.593530 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:33.602398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:33.610268 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a7f9e-e9aa-4130-9f74-4dd9e746d532 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:33.618949 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6519c-ee07-4011-a4dd-46d1f107ab33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:33.620095 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6519c-ee07-4011-a4dd-46d1f107ab33 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:33.620286 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6519c-ee07-4011-a4dd-46d1f107ab33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:33.621199 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6519c-ee07-4011-a4dd-46d1f107ab33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:33.622002 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e6519c-ee07-4011-a4dd-46d1f107ab33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:33.625127 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e6519c-ee07-4011-a4dd-46d1f107ab33 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:33.683088 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1032/2045] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:33 2022] POST /identity/v3/projects => generated 383 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:33.689672 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca78b48-fb16-4d50-a55a-c6677b06a774 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:33.692002 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca78b48-fb16-4d50-a55a-c6677b06a774 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:33.693150 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca78b48-fb16-4d50-a55a-c6677b06a774 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:33.693371 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6519c-ee07-4011-a4dd-46d1f107ab33 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:33.694961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca78b48-fb16-4d50-a55a-c6677b06a774 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:33.699086 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-eca78b48-fb16-4d50-a55a-c6677b06a774 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:33.811330 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6519c-ee07-4011-a4dd-46d1f107ab33 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:33.817298 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f424254-52d4-48a8-9895-53b09debbdb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:33.818127 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f424254-52d4-48a8-9895-53b09debbdb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:33.818421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f424254-52d4-48a8-9895-53b09debbdb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:33.822176 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f424254-52d4-48a8-9895-53b09debbdb5 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:33.823033 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f424254-52d4-48a8-9895-53b09debbdb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:33.825076 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f424254-52d4-48a8-9895-53b09debbdb5 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:33.865548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1033/2046] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:33 2022] POST /identity/v3/users => generated 292 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:33.873534 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f424254-52d4-48a8-9895-53b09debbdb5 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:33.882970 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca78b48-fb16-4d50-a55a-c6677b06a774 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:33.886640 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca78b48-fb16-4d50-a55a-c6677b06a774 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:33.892134 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f424254-52d4-48a8-9895-53b09debbdb5 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:33.901340 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728892f7-b14f-4a9d-aeff-ee81264cc09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:33.902520 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728892f7-b14f-4a9d-aeff-ee81264cc09c admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:33.902855 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728892f7-b14f-4a9d-aeff-ee81264cc09c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:33.903132 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728892f7-b14f-4a9d-aeff-ee81264cc09c admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:33.903917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728892f7-b14f-4a9d-aeff-ee81264cc09c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:33.906402 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728892f7-b14f-4a9d-aeff-ee81264cc09c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:33.906692 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1014/2047] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:33 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:33.924158 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1034/2048] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:33.934624 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca78b48-fb16-4d50-a55a-c6677b06a774 service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:33.936880 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-eca78b48-fb16-4d50-a55a-c6677b06a774 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:33.939299 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728892f7-b14f-4a9d-aeff-ee81264cc09c admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:33.961233 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca78b48-fb16-4d50-a55a-c6677b06a774 service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:33.971971 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728892f7-b14f-4a9d-aeff-ee81264cc09c admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:33.988656 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea72214-f2c0-49bc-aac2-46b95b502aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:33.989345 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea72214-f2c0-49bc-aac2-46b95b502aa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:33.989631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea72214-f2c0-49bc-aac2-46b95b502aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:33.989921 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea72214-f2c0-49bc-aac2-46b95b502aa0 admin admin] PATH_INFO: `/v3/projects/f05c0f7bf505463dae8da1318f2a1d6a` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:33.991594 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea72214-f2c0-49bc-aac2-46b95b502aa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f05c0f7bf505463dae8da1318f2a1d6a)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:33.994198 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea72214-f2c0-49bc-aac2-46b95b502aa0 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:33.995099 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c8c27-8752-45b0-bc51-c6057007697c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:33.996193 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c8c27-8752-45b0-bc51-c6057007697c admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:33.996750 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c8c27-8752-45b0-bc51-c6057007697c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:33.999418 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c8c27-8752-45b0-bc51-c6057007697c admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:34.000998 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2c8c27-8752-45b0-bc51-c6057007697c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:34.005231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2c8c27-8752-45b0-bc51-c6057007697c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:34.031760 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1035/2049] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:34.033264 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c8c27-8752-45b0-bc51-c6057007697c admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:34.050897 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c8c27-8752-45b0-bc51-c6057007697c admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:34.080884 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-bea72214-f2c0-49bc-aac2-46b95b502aa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f05c0f7bf505463dae8da1318f2a1d6a', 'request_id': 'req-bea72214-f2c0-49bc-aac2-46b95b502aa0'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:34.092744 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-bea72214-f2c0-49bc-aac2-46b95b502aa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f05c0f7bf505463dae8da1318f2a1d6a', 'request_id': 'req-bea72214-f2c0-49bc-aac2-46b95b502aa0'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:34.107568 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-bea72214-f2c0-49bc-aac2-46b95b502aa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f05c0f7bf505463dae8da1318f2a1d6a', 'request_id': 'req-bea72214-f2c0-49bc-aac2-46b95b502aa0'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:34.125501 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-bea72214-f2c0-49bc-aac2-46b95b502aa0 admin admin] The token cache is being invalidate because project f05c0f7bf505463dae8da1318f2a1d6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:34.125792 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-bea72214-f2c0-49bc-aac2-46b95b502aa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:34.126543 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-bea72214-f2c0-49bc-aac2-46b95b502aa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:34.135689 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1015/2050] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:33 2022] DELETE /identity/v3/projects/f05c0f7bf505463dae8da1318f2a1d6a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:34.137798 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31768197-e671-4c14-af2a-b4b1b94f69aa None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:34.138455 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31768197-e671-4c14-af2a-b4b1b94f69aa None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:34.138707 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31768197-e671-4c14-af2a-b4b1b94f69aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:34.152792 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea557078-1553-4708-ab5c-4ea4b36b294f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:34.154227 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea557078-1553-4708-ab5c-4ea4b36b294f admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:34.154743 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea557078-1553-4708-ab5c-4ea4b36b294f admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:34.155252 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea557078-1553-4708-ab5c-4ea4b36b294f admin admin] PATH_INFO: `/v3/projects/92d3a03f469746a1b8f379f2a61069ea/users/402e73b674d44334b83ce2e8d4969a10/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:34.172137 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-31768197-e671-4c14-af2a-b4b1b94f69aa None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:34.174292 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-31768197-e671-4c14-af2a-b4b1b94f69aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:34.174407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea557078-1553-4708-ab5c-4ea4b36b294f admin admin] RBAC: Authorizing `identity:create_grant(project_id=92d3a03f469746a1b8f379f2a61069ea, user_id=402e73b674d44334b83ce2e8d4969a10, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:34.178508 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea557078-1553-4708-ab5c-4ea4b36b294f admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:34.184873 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1016/2051] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:34.197411 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1036/2052] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:34 2022] PUT /identity/v3/projects/92d3a03f469746a1b8f379f2a61069ea/users/402e73b674d44334b83ce2e8d4969a10/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:34.204663 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba91111-2fdb-42ef-aada-81e82c9e5de3 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:34.205061 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba91111-2fdb-42ef-aada-81e82c9e5de3 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:34.205434 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba91111-2fdb-42ef-aada-81e82c9e5de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:34.208550 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400287b-b311-40f4-af86-850db4380663 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:34.208844 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400287b-b311-40f4-af86-850db4380663 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:34.209125 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400287b-b311-40f4-af86-850db4380663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:34.229593 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-aba91111-2fdb-42ef-aada-81e82c9e5de3 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:34.230826 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-aba91111-2fdb-42ef-aada-81e82c9e5de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:34.244707 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-2400287b-b311-40f4-af86-850db4380663 None None] MFA Rules not processed for user `402e73b674d44334b83ce2e8d4969a10`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:34.246024 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2400287b-b311-40f4-af86-850db4380663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:34.270842 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:34.288168 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:34.314025 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1017/2053] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:22:34.315753 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba91111-2fdb-42ef-aada-81e82c9e5de3 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:34.327597 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba91111-2fdb-42ef-aada-81e82c9e5de3 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:34.331061 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1037/2054] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:22:34.333695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b9464-64c3-4252-96a6-a28bca78ed44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:34.334558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b9464-64c3-4252-96a6-a28bca78ed44 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:34.335166 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b9464-64c3-4252-96a6-a28bca78ed44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:34.335475 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b9464-64c3-4252-96a6-a28bca78ed44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:34.336030 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400287b-b311-40f4-af86-850db4380663 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:34.336226 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b9464-64c3-4252-96a6-a28bca78ed44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:22:34.336533 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b9464-64c3-4252-96a6-a28bca78ed44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:34.337513 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2400287b-b311-40f4-af86-850db4380663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:34.338398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b9464-64c3-4252-96a6-a28bca78ed44 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:34.350838 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1018/2055] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:22:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:22:34.353653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400287b-b311-40f4-af86-850db4380663 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:34.355622 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b9464-64c3-4252-96a6-a28bca78ed44 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:34.358921 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7da54e-6d2e-4298-8069-e78e3ac5fa52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:34.359685 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7da54e-6d2e-4298-8069-e78e3ac5fa52 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:34.359977 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7da54e-6d2e-4298-8069-e78e3ac5fa52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:34.360258 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7da54e-6d2e-4298-8069-e78e3ac5fa52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:34.360941 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7da54e-6d2e-4298-8069-e78e3ac5fa52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:34.362892 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7da54e-6d2e-4298-8069-e78e3ac5fa52 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:34.365667 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b9464-64c3-4252-96a6-a28bca78ed44 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:34.371224 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d20e39-2e36-4300-83ce-c8bfe3a9d347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:34.371911 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d20e39-2e36-4300-83ce-c8bfe3a9d347 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:34.372260 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d20e39-2e36-4300-83ce-c8bfe3a9d347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:34.372602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d20e39-2e36-4300-83ce-c8bfe3a9d347 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:34.373365 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d20e39-2e36-4300-83ce-c8bfe3a9d347 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:34.375543 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d20e39-2e36-4300-83ce-c8bfe3a9d347 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:34.405931 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1019/2056] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:34 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:22:34.409056 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d20e39-2e36-4300-83ce-c8bfe3a9d347 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:34.410823 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-15d20e39-2e36-4300-83ce-c8bfe3a9d347 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:34.411965 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1038/2057] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:34 2022] POST /identity/v3/projects => generated 383 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:22:34.412608 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7da54e-6d2e-4298-8069-e78e3ac5fa52 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:34.482043 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d20e39-2e36-4300-83ce-c8bfe3a9d347 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:34.486417 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7da54e-6d2e-4298-8069-e78e3ac5fa52 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:34.491264 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26fccf1-7811-48c0-bf40-3522e21dccfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:34.491881 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fccf1-7811-48c0-bf40-3522e21dccfe admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:34.492164 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fccf1-7811-48c0-bf40-3522e21dccfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:34.492454 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fccf1-7811-48c0-bf40-3522e21dccfe admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:34.493109 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26fccf1-7811-48c0-bf40-3522e21dccfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:34.495239 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26fccf1-7811-48c0-bf40-3522e21dccfe admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:34.517903 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1039/2058] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:34 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:34.518574 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26fccf1-7811-48c0-bf40-3522e21dccfe admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:34.528738 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26fccf1-7811-48c0-bf40-3522e21dccfe admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:34.533612 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10de6a1-f330-4e0b-9ef5-5baae943be54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:34.534236 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10de6a1-f330-4e0b-9ef5-5baae943be54 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:34.534519 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10de6a1-f330-4e0b-9ef5-5baae943be54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:34.534791 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10de6a1-f330-4e0b-9ef5-5baae943be54 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:34.535446 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10de6a1-f330-4e0b-9ef5-5baae943be54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:34.537352 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10de6a1-f330-4e0b-9ef5-5baae943be54 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:34.540815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008db27-214d-47a9-ab47-f67ad99ed7cc tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01fc37aab0c407d9ca6aeecff422159', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b594579fd4534918acf5fd1e38cff36b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:34.541444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008db27-214d-47a9-ab47-f67ad99ed7cc tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:34.541733 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008db27-214d-47a9-ab47-f67ad99ed7cc tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:34.542183 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008db27-214d-47a9-ab47-f67ad99ed7cc tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:34.542873 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6008db27-214d-47a9-ab47-f67ad99ed7cc tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:22:34.543178 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6008db27-214d-47a9-ab47-f67ad99ed7cc tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:34.544789 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6008db27-214d-47a9-ab47-f67ad99ed7cc tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:34.557596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1020/2059] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:22:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:22:34.561923 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1040/2060] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:34 2022] POST /identity/v3/users => generated 298 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:34.562101 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008db27-214d-47a9-ab47-f67ad99ed7cc tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:34.562876 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10de6a1-f330-4e0b-9ef5-5baae943be54 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:34.573219 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008db27-214d-47a9-ab47-f67ad99ed7cc tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:34.574450 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10de6a1-f330-4e0b-9ef5-5baae943be54 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:34.577866 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f569b7-f199-4162-9d0b-b4db7609e222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:34.578496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f569b7-f199-4162-9d0b-b4db7609e222 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:34.578777 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f569b7-f199-4162-9d0b-b4db7609e222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:34.579389 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f569b7-f199-4162-9d0b-b4db7609e222 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:34.579740 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f569b7-f199-4162-9d0b-b4db7609e222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:34.581322 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f569b7-f199-4162-9d0b-b4db7609e222 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:34.584793 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4dfc83-76e1-4012-a3bf-1f923b8da33d tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01fc37aab0c407d9ca6aeecff422159', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b594579fd4534918acf5fd1e38cff36b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:34.585416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4dfc83-76e1-4012-a3bf-1f923b8da33d tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:34.585701 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4dfc83-76e1-4012-a3bf-1f923b8da33d tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:34.585992 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4dfc83-76e1-4012-a3bf-1f923b8da33d tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:34.587240 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4dfc83-76e1-4012-a3bf-1f923b8da33d tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:34.589276 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4dfc83-76e1-4012-a3bf-1f923b8da33d tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:34.592449 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1021/2061] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:34.596281 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f569b7-f199-4162-9d0b-b4db7609e222 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:34.606700 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f569b7-f199-4162-9d0b-b4db7609e222 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:34.611820 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b943a-d262-4463-ad47-45ee758bb5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:34.612443 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b943a-d262-4463-ad47-45ee758bb5da admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:34.612753 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b943a-d262-4463-ad47-45ee758bb5da admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:34.613191 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b943a-d262-4463-ad47-45ee758bb5da admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:34.613899 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b943a-d262-4463-ad47-45ee758bb5da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:34.615607 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b943a-d262-4463-ad47-45ee758bb5da admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:34.625527 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1041/2062] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:34 2022] POST /identity/v3/projects => generated 354 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:22:34.626369 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4dfc83-76e1-4012-a3bf-1f923b8da33d tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:34.635644 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1022/2063] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:34.635644 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b943a-d262-4463-ad47-45ee758bb5da admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:34.651370 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b943a-d262-4463-ad47-45ee758bb5da admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:34.690898 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:34.701346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:34.724673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4dfc83-76e1-4012-a3bf-1f923b8da33d tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:34.728065 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a9d12-d176-4e23-9b4a-f7d04a19f68b tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01fc37aab0c407d9ca6aeecff422159', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b594579fd4534918acf5fd1e38cff36b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:34.729003 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a9d12-d176-4e23-9b4a-f7d04a19f68b tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:34.729297 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a9d12-d176-4e23-9b4a-f7d04a19f68b tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:34.729550 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a9d12-d176-4e23-9b4a-f7d04a19f68b tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:34.730386 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ac68e-f83f-4ca0-ae0a-f9473c542da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:34.730512 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81a9d12-d176-4e23-9b4a-f7d04a19f68b tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:22:34.730610 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ac68e-f83f-4ca0-ae0a-f9473c542da3 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:34.730688 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81a9d12-d176-4e23-9b4a-f7d04a19f68b tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:34.730873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ac68e-f83f-4ca0-ae0a-f9473c542da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:34.731240 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ac68e-f83f-4ca0-ae0a-f9473c542da3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:34.732192 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81a9d12-d176-4e23-9b4a-f7d04a19f68b tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:34.732421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ac68e-f83f-4ca0-ae0a-f9473c542da3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:34.734344 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ac68e-f83f-4ca0-ae0a-f9473c542da3 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:34.747085 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1042/2064] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:34.747951 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ac68e-f83f-4ca0-ae0a-f9473c542da3 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:34.751256 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1023/2065] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:22:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:22:34.753080 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a9d12-d176-4e23-9b4a-f7d04a19f68b tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:34.760787 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ac68e-f83f-4ca0-ae0a-f9473c542da3 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:34.766955 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a9d12-d176-4e23-9b4a-f7d04a19f68b tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:34.767235 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1564f7-7f9e-4782-90e4-c5e49ca4b265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:34.768676 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1564f7-7f9e-4782-90e4-c5e49ca4b265 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:34.769237 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1564f7-7f9e-4782-90e4-c5e49ca4b265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:34.769863 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1564f7-7f9e-4782-90e4-c5e49ca4b265 admin admin] PATH_INFO: `/v3/projects/70e1ee59339844b98a4c4fe41127e330/users/2b8caa5cdc31471fa7ad04c2fd0ab286/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:34.778741 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6050ef2e-8227-42ac-a346-f0dba328033a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:34.779442 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050ef2e-8227-42ac-a346-f0dba328033a admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:34.779923 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050ef2e-8227-42ac-a346-f0dba328033a admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:34.780554 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050ef2e-8227-42ac-a346-f0dba328033a admin admin] PATH_INFO: `/v3/projects/36a8c698b177483ab558e0d4afc8ee5d/users/9935fdc20abb4fc09ffb0ea7e055226c/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:34.794295 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1564f7-7f9e-4782-90e4-c5e49ca4b265 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70e1ee59339844b98a4c4fe41127e330, user_id=2b8caa5cdc31471fa7ad04c2fd0ab286, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:34.801658 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1564f7-7f9e-4782-90e4-c5e49ca4b265 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:34.808394 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6050ef2e-8227-42ac-a346-f0dba328033a admin admin] RBAC: Authorizing `identity:create_grant(project_id=36a8c698b177483ab558e0d4afc8ee5d, user_id=9935fdc20abb4fc09ffb0ea7e055226c, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:34.811578 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6050ef2e-8227-42ac-a346-f0dba328033a admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:34.822559 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1043/2066] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:34 2022] PUT /identity/v3/projects/70e1ee59339844b98a4c4fe41127e330/users/2b8caa5cdc31471fa7ad04c2fd0ab286/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:34.823480 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1564f7-7f9e-4782-90e4-c5e49ca4b265 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:34.827840 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1024/2067] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:34 2022] PUT /identity/v3/projects/36a8c698b177483ab558e0d4afc8ee5d/users/9935fdc20abb4fc09ffb0ea7e055226c/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:34.828675 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6050ef2e-8227-42ac-a346-f0dba328033a admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:34.894593 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1564f7-7f9e-4782-90e4-c5e49ca4b265 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:34.916172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6050ef2e-8227-42ac-a346-f0dba328033a admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:34.922727 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c026a41-0e3b-4429-a05e-ccdf10070a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:34.923426 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c026a41-0e3b-4429-a05e-ccdf10070a65 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:34.924102 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c026a41-0e3b-4429-a05e-ccdf10070a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:34.924380 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c026a41-0e3b-4429-a05e-ccdf10070a65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:34.925097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c026a41-0e3b-4429-a05e-ccdf10070a65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:34.926851 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c026a41-0e3b-4429-a05e-ccdf10070a65 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:34.940701 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1025/2068] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:34.942013 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455d0f7-8f6c-4e8c-8192-580e87407e55 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:34.942307 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455d0f7-8f6c-4e8c-8192-580e87407e55 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:34.942593 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455d0f7-8f6c-4e8c-8192-580e87407e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:34.962771 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9844d2cb-a48f-4d12-ae9e-4ecc783a8db7 tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b01fc37aab0c407d9ca6aeecff422159', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b594579fd4534918acf5fd1e38cff36b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:34.963603 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844d2cb-a48f-4d12-ae9e-4ecc783a8db7 tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:34.963957 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844d2cb-a48f-4d12-ae9e-4ecc783a8db7 tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:34.964337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844d2cb-a48f-4d12-ae9e-4ecc783a8db7 tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] PATH_INFO: `/v3/projects/aeb608aa518c4d19961d43f9646a95ee` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:34.965682 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9844d2cb-a48f-4d12-ae9e-4ecc783a8db7 tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] RBAC: Authorizing `identity:delete_project(project_id=aeb608aa518c4d19961d43f9646a95ee)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:34.967725 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9844d2cb-a48f-4d12-ae9e-4ecc783a8db7 tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:34.979217 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-4455d0f7-8f6c-4e8c-8192-580e87407e55 None None] MFA Rules not processed for user `9935fdc20abb4fc09ffb0ea7e055226c`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:34.980581 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-4455d0f7-8f6c-4e8c-8192-580e87407e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:35.021579 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9844d2cb-a48f-4d12-ae9e-4ecc783a8db7 tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aeb608aa518c4d19961d43f9646a95ee', 'request_id': 'req-9844d2cb-a48f-4d12-ae9e-4ecc783a8db7'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:35.029218 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9844d2cb-a48f-4d12-ae9e-4ecc783a8db7 tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aeb608aa518c4d19961d43f9646a95ee', 'request_id': 'req-9844d2cb-a48f-4d12-ae9e-4ecc783a8db7'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:35.038765 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9844d2cb-a48f-4d12-ae9e-4ecc783a8db7 tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aeb608aa518c4d19961d43f9646a95ee', 'request_id': 'req-9844d2cb-a48f-4d12-ae9e-4ecc783a8db7'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:35.044504 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:35.059813 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9844d2cb-a48f-4d12-ae9e-4ecc783a8db7 tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] The token cache is being invalidate because project aeb608aa518c4d19961d43f9646a95ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:35.060233 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9844d2cb-a48f-4d12-ae9e-4ecc783a8db7 tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:35.061038 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9844d2cb-a48f-4d12-ae9e-4ecc783a8db7 tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:35.061836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1026/2069] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:35.062692 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4455d0f7-8f6c-4e8c-8192-580e87407e55 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:35.062791 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1044/2070] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:34 2022] DELETE /identity/v3/projects/aeb608aa518c4d19961d43f9646a95ee => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:35.066462 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9844d2cb-a48f-4d12-ae9e-4ecc783a8db7 tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:35.067890 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-9844d2cb-a48f-4d12-ae9e-4ecc783a8db7 tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:35.074933 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4455d0f7-8f6c-4e8c-8192-580e87407e55 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:35.106249 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:35.143346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9844d2cb-a48f-4d12-ae9e-4ecc783a8db7 tempest-VolumeQuotaClassesTest-829437868 tempest-VolumeQuotaClassesTest-829437868-project-admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:35.148406 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fba84-d475-41b6-a44e-065d2ef99ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:35.149176 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fba84-d475-41b6-a44e-065d2ef99ad0 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:35.149285 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef349a-2166-4324-971d-dfe0306e7311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:35.149490 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fba84-d475-41b6-a44e-065d2ef99ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:35.150536 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef349a-2166-4324-971d-dfe0306e7311 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:35.150656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fba84-d475-41b6-a44e-065d2ef99ad0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:35.150838 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef349a-2166-4324-971d-dfe0306e7311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:35.151086 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef349a-2166-4324-971d-dfe0306e7311 admin admin] PATH_INFO: `/v3/projects/70e1ee59339844b98a4c4fe41127e330/users/2b8caa5cdc31471fa7ad04c2fd0ab286/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:35.151283 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366fba84-d475-41b6-a44e-065d2ef99ad0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:35.153898 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ef349a-2166-4324-971d-dfe0306e7311 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70e1ee59339844b98a4c4fe41127e330, user_id=2b8caa5cdc31471fa7ad04c2fd0ab286, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:35.154060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366fba84-d475-41b6-a44e-065d2ef99ad0 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:35.156841 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ef349a-2166-4324-971d-dfe0306e7311 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:35.168886 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1027/2071] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:35 2022] PUT /identity/v3/projects/70e1ee59339844b98a4c4fe41127e330/users/2b8caa5cdc31471fa7ad04c2fd0ab286/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:35.178930 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b7ae2-8933-41c4-a228-51c35614e567 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:35.179141 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b7ae2-8933-41c4-a228-51c35614e567 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:35.179585 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b7ae2-8933-41c4-a228-51c35614e567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:35.188840 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1045/2072] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:35 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:22:35.189898 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fba84-d475-41b6-a44e-065d2ef99ad0 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:35.216290 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-0e7b7ae2-8933-41c4-a228-51c35614e567 None None] MFA Rules not processed for user `2b8caa5cdc31471fa7ad04c2fd0ab286`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:35.218109 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0e7b7ae2-8933-41c4-a228-51c35614e567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:35.297413 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fba84-d475-41b6-a44e-065d2ef99ad0 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:35.303857 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3351d6-0281-479a-bfbb-5a9115b1419e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:35.305054 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3351d6-0281-479a-bfbb-5a9115b1419e admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:35.305337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3351d6-0281-479a-bfbb-5a9115b1419e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:35.305610 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3351d6-0281-479a-bfbb-5a9115b1419e admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:35.306996 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3351d6-0281-479a-bfbb-5a9115b1419e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:35.309153 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3351d6-0281-479a-bfbb-5a9115b1419e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:35.324151 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1028/2073] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:35 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:22:35.326926 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b7ae2-8933-41c4-a228-51c35614e567 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:35.328461 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0e7b7ae2-8933-41c4-a228-51c35614e567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:35.338010 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1046/2074] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:35 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:22:35.338010 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3351d6-0281-479a-bfbb-5a9115b1419e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:35.338355 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-af3351d6-0281-479a-bfbb-5a9115b1419e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:35.343367 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b7ae2-8933-41c4-a228-51c35614e567 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:35.348261 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95c276-9907-40ad-9746-0053f20f9b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:35.348989 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95c276-9907-40ad-9746-0053f20f9b9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:35.349298 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95c276-9907-40ad-9746-0053f20f9b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:35.349574 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95c276-9907-40ad-9746-0053f20f9b9e admin admin] PATH_INFO: `/v3/users/ec2fda6d3b744369a27bbb57a7b61232` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:35.355268 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a95c276-9907-40ad-9746-0053f20f9b9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec2fda6d3b744369a27bbb57a7b61232)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:35.357493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a95c276-9907-40ad-9746-0053f20f9b9e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:35.406773 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3351d6-0281-479a-bfbb-5a9115b1419e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:35.414263 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d0a95-e1d0-462b-866d-8a065bc35715 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:35.414900 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d0a95-e1d0-462b-866d-8a065bc35715 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:35.415771 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d0a95-e1d0-462b-866d-8a065bc35715 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:35.416060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d0a95-e1d0-462b-866d-8a065bc35715 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:35.422008 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2a95c276-9907-40ad-9746-0053f20f9b9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec2fda6d3b744369a27bbb57a7b61232'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:35.422739 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2a95c276-9907-40ad-9746-0053f20f9b9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec2fda6d3b744369a27bbb57a7b61232'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:35.425714 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d0a95-e1d0-462b-866d-8a065bc35715 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:35.427189 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d0a95-e1d0-462b-866d-8a065bc35715 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:35.429814 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2a95c276-9907-40ad-9746-0053f20f9b9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec2fda6d3b744369a27bbb57a7b61232'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:35.437430 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1047/2075] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:35 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:35.438328 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d0a95-e1d0-462b-866d-8a065bc35715 service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:35.442450 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2a95c276-9907-40ad-9746-0053f20f9b9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec2fda6d3b744369a27bbb57a7b61232'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:35.449303 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d0a95-e1d0-462b-866d-8a065bc35715 service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:35.452203 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2a95c276-9907-40ad-9746-0053f20f9b9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec2fda6d3b744369a27bbb57a7b61232'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:35.460900 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f775c-6f85-4c6a-a477-e28a4d884d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:35.461529 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f775c-6f85-4c6a-a477-e28a4d884d24 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:35.461837 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f775c-6f85-4c6a-a477-e28a4d884d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:35.467079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f775c-6f85-4c6a-a477-e28a4d884d24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:35.467079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9f775c-6f85-4c6a-a477-e28a4d884d24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:35.467079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9f775c-6f85-4c6a-a477-e28a4d884d24 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:35.473103 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1029/2076] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:35 2022] DELETE /identity/v3/users/ec2fda6d3b744369a27bbb57a7b61232 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:35.474620 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8e1c4-e6ef-436b-9020-5d3fe66c432a None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:35.474963 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8e1c4-e6ef-436b-9020-5d3fe66c432a None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:35.475306 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8e1c4-e6ef-436b-9020-5d3fe66c432a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:35.478516 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1048/2077] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:35.485415 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f775c-6f85-4c6a-a477-e28a4d884d24 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:35.487779 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5b9f775c-6f85-4c6a-a477-e28a4d884d24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:35.497805 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:35.503458 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f775c-6f85-4c6a-a477-e28a4d884d24 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:35.504054 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-def8e1c4-e6ef-436b-9020-5d3fe66c432a None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:35.505384 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-def8e1c4-e6ef-436b-9020-5d3fe66c432a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:35.521220 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:35.575491 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb7d496-b06f-4e78-aef9-654de135f6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:35.576172 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7d496-b06f-4e78-aef9-654de135f6e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:35.576455 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7d496-b06f-4e78-aef9-654de135f6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:35.576735 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7d496-b06f-4e78-aef9-654de135f6e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:35.577428 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb7d496-b06f-4e78-aef9-654de135f6e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:35.579069 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb7d496-b06f-4e78-aef9-654de135f6e7 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:35.581184 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1030/2078] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:35.582043 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8e1c4-e6ef-436b-9020-5d3fe66c432a None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:35.583297 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-def8e1c4-e6ef-436b-9020-5d3fe66c432a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:35.594048 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1049/2079] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:35.595034 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb7d496-b06f-4e78-aef9-654de135f6e7 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:35.598130 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-4cb7d496-b06f-4e78-aef9-654de135f6e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:35.647367 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8e1c4-e6ef-436b-9020-5d3fe66c432a None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:35.651301 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a5dc3-a44e-427f-847b-6cb1609f056d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:35.651921 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a5dc3-a44e-427f-847b-6cb1609f056d service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:35.652198 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a5dc3-a44e-427f-847b-6cb1609f056d service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:35.652468 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a5dc3-a44e-427f-847b-6cb1609f056d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:35.654295 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c56a5dc3-a44e-427f-847b-6cb1609f056d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:35.663970 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb7d496-b06f-4e78-aef9-654de135f6e7 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:35.668528 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799dff67-1ca4-4135-af1a-5b51f301a446 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:35.669156 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799dff67-1ca4-4135-af1a-5b51f301a446 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:35.669447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799dff67-1ca4-4135-af1a-5b51f301a446 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:35.669728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799dff67-1ca4-4135-af1a-5b51f301a446 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:35.679586 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799dff67-1ca4-4135-af1a-5b51f301a446 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:35.681957 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799dff67-1ca4-4135-af1a-5b51f301a446 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:35.698122 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1050/2080] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:35.698939 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799dff67-1ca4-4135-af1a-5b51f301a446 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:35.709006 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799dff67-1ca4-4135-af1a-5b51f301a446 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:35.716623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e937524-4ee6-4e02-ae83-f1e5d7cd14f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:35.717305 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e937524-4ee6-4e02-ae83-f1e5d7cd14f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:35.717585 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e937524-4ee6-4e02-ae83-f1e5d7cd14f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:35.717861 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e937524-4ee6-4e02-ae83-f1e5d7cd14f2 admin admin] PATH_INFO: `/v3/projects/2bd4c85c2d4b4a9a8f0e89defe7d8a20/users/b894de5959714790b592c65f0afc8202/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:35.764858 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e937524-4ee6-4e02-ae83-f1e5d7cd14f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bd4c85c2d4b4a9a8f0e89defe7d8a20, user_id=b894de5959714790b592c65f0afc8202, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:35.771943 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e937524-4ee6-4e02-ae83-f1e5d7cd14f2 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:35.787959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56a5dc3-a44e-427f-847b-6cb1609f056d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:35.795582 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56a5dc3-a44e-427f-847b-6cb1609f056d service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:35.800465 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1051/2081] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:35 2022] PUT /identity/v3/projects/2bd4c85c2d4b4a9a8f0e89defe7d8a20/users/b894de5959714790b592c65f0afc8202/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:35.813931 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe63cd-c5ae-441f-a3ca-14cfb8a1702a None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:35.817559 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe63cd-c5ae-441f-a3ca-14cfb8a1702a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:35.817904 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe63cd-c5ae-441f-a3ca-14cfb8a1702a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:35.836701 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1031/2082] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:35.860333 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-00fe63cd-c5ae-441f-a3ca-14cfb8a1702a None None] MFA Rules not processed for user `b894de5959714790b592c65f0afc8202`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:35.861880 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-00fe63cd-c5ae-441f-a3ca-14cfb8a1702a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:35.979427 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1052/2083] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:35.983814 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a5dc3-a44e-427f-847b-6cb1609f056d service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:36.001645 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a5dc3-a44e-427f-847b-6cb1609f056d service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:36.158930 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad97564-6805-4840-b10b-424642662c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:36.160201 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad97564-6805-4840-b10b-424642662c2e admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:36.160493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad97564-6805-4840-b10b-424642662c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:36.160779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad97564-6805-4840-b10b-424642662c2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:36.161653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad97564-6805-4840-b10b-424642662c2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:36.163633 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad97564-6805-4840-b10b-424642662c2e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:36.197889 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:36.211389 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1032/2084] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:35 2022] POST /identity/v3/projects => generated 387 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:36.218198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe63cd-c5ae-441f-a3ca-14cfb8a1702a None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:36.236603 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe63cd-c5ae-441f-a3ca-14cfb8a1702a None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:36.242955 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:36.361114 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b44547a-a053-47bc-bd3e-caa19b9cdeb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:36.361755 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b44547a-a053-47bc-bd3e-caa19b9cdeb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:36.362127 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b44547a-a053-47bc-bd3e-caa19b9cdeb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:36.362363 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b44547a-a053-47bc-bd3e-caa19b9cdeb5 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:36.363060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b44547a-a053-47bc-bd3e-caa19b9cdeb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:36.365010 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b44547a-a053-47bc-bd3e-caa19b9cdeb5 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:36.402088 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1053/2085] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:36 2022] POST /identity/v3/users => generated 300 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:36.403222 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad97564-6805-4840-b10b-424642662c2e admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:36.416702 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad97564-6805-4840-b10b-424642662c2e admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:36.419757 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b44547a-a053-47bc-bd3e-caa19b9cdeb5 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:36.423077 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5b44547a-a053-47bc-bd3e-caa19b9cdeb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:36.428381 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8715742-3a33-4e65-bdf1-c088299e1eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:36.430343 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8715742-3a33-4e65-bdf1-c088299e1eaf admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:36.430635 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8715742-3a33-4e65-bdf1-c088299e1eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:36.431642 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8715742-3a33-4e65-bdf1-c088299e1eaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:36.433359 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8715742-3a33-4e65-bdf1-c088299e1eaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:36.435959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8715742-3a33-4e65-bdf1-c088299e1eaf admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:36.440393 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b44547a-a053-47bc-bd3e-caa19b9cdeb5 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:36.448498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3602c03a-05a7-40d1-a990-e5f20a1ac9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:36.449282 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602c03a-05a7-40d1-a990-e5f20a1ac9f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:36.449813 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602c03a-05a7-40d1-a990-e5f20a1ac9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:36.451319 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602c03a-05a7-40d1-a990-e5f20a1ac9f9 admin admin] PATH_INFO: `/v3/projects/192024afe10f477bb8cecd9fe1f44d14` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:36.452310 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3602c03a-05a7-40d1-a990-e5f20a1ac9f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=192024afe10f477bb8cecd9fe1f44d14)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:36.456526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3602c03a-05a7-40d1-a990-e5f20a1ac9f9 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:36.463372 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1033/2086] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 04:22:36.468216 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8715742-3a33-4e65-bdf1-c088299e1eaf admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:36.492261 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8715742-3a33-4e65-bdf1-c088299e1eaf admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:36.500053 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b601e-99d9-408c-85d6-329cdac371b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:36.501223 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b601e-99d9-408c-85d6-329cdac371b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:36.501518 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b601e-99d9-408c-85d6-329cdac371b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:36.502697 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b601e-99d9-408c-85d6-329cdac371b4 admin admin] PATH_INFO: `/v3/projects/e7987650d9ee4c74950fd33efd9f2514/users/615c6cdc808b489f80c7dc53c3ace941/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:36.525526 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b601e-99d9-408c-85d6-329cdac371b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7987650d9ee4c74950fd33efd9f2514, user_id=615c6cdc808b489f80c7dc53c3ace941, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:36.528588 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b601e-99d9-408c-85d6-329cdac371b4 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:36.543962 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3602c03a-05a7-40d1-a990-e5f20a1ac9f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '192024afe10f477bb8cecd9fe1f44d14', 'request_id': 'req-3602c03a-05a7-40d1-a990-e5f20a1ac9f9'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:36.550280 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1034/2087] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:36 2022] PUT /identity/v3/projects/e7987650d9ee4c74950fd33efd9f2514/users/615c6cdc808b489f80c7dc53c3ace941/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:22:36.553162 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b601e-99d9-408c-85d6-329cdac371b4 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:36.556374 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3602c03a-05a7-40d1-a990-e5f20a1ac9f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '192024afe10f477bb8cecd9fe1f44d14', 'request_id': 'req-3602c03a-05a7-40d1-a990-e5f20a1ac9f9'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:36.563843 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3602c03a-05a7-40d1-a990-e5f20a1ac9f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '192024afe10f477bb8cecd9fe1f44d14', 'request_id': 'req-3602c03a-05a7-40d1-a990-e5f20a1ac9f9'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:36.581669 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3602c03a-05a7-40d1-a990-e5f20a1ac9f9 admin admin] The token cache is being invalidate because project 192024afe10f477bb8cecd9fe1f44d14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:36.582186 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3602c03a-05a7-40d1-a990-e5f20a1ac9f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:36.583210 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3602c03a-05a7-40d1-a990-e5f20a1ac9f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:36.585089 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1054/2088] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:36 2022] DELETE /identity/v3/projects/192024afe10f477bb8cecd9fe1f44d14 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:36.589067 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3602c03a-05a7-40d1-a990-e5f20a1ac9f9 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:36.591035 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-3602c03a-05a7-40d1-a990-e5f20a1ac9f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:36.661842 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b601e-99d9-408c-85d6-329cdac371b4 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:36.668197 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3602c03a-05a7-40d1-a990-e5f20a1ac9f9 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:36.682334 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54656a8-f63b-4cd8-aba9-faff31480410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:36.683092 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54656a8-f63b-4cd8-aba9-faff31480410 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:36.683399 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54656a8-f63b-4cd8-aba9-faff31480410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:36.683646 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54656a8-f63b-4cd8-aba9-faff31480410 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:36.684354 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54656a8-f63b-4cd8-aba9-faff31480410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:36.685987 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54656a8-f63b-4cd8-aba9-faff31480410 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:36.686122 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec4716-451a-4094-8754-c8e56b0d907e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:36.686814 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec4716-451a-4094-8754-c8e56b0d907e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:36.687247 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec4716-451a-4094-8754-c8e56b0d907e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:36.687662 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec4716-451a-4094-8754-c8e56b0d907e admin admin] PATH_INFO: `/v3/users/b01fc37aab0c407d9ca6aeecff422159` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:36.688798 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec4716-451a-4094-8754-c8e56b0d907e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b01fc37aab0c407d9ca6aeecff422159)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:36.691713 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec4716-451a-4094-8754-c8e56b0d907e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:36.698812 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1035/2089] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 04:22:36.702705 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54656a8-f63b-4cd8-aba9-faff31480410 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:36.704343 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c54656a8-f63b-4cd8-aba9-faff31480410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:36.718517 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54656a8-f63b-4cd8-aba9-faff31480410 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:36.719729 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:36.723616 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e06e7d-e569-4778-9147-da1e5f244ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:36.724291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e06e7d-e569-4778-9147-da1e5f244ace admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:36.724575 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e06e7d-e569-4778-9147-da1e5f244ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:36.724946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e06e7d-e569-4778-9147-da1e5f244ace admin admin] PATH_INFO: `/v3/projects/e7987650d9ee4c74950fd33efd9f2514/users/615c6cdc808b489f80c7dc53c3ace941/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:36.727065 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e06e7d-e569-4778-9147-da1e5f244ace admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7987650d9ee4c74950fd33efd9f2514, user_id=615c6cdc808b489f80c7dc53c3ace941, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:36.731442 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e06e7d-e569-4778-9147-da1e5f244ace admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:36.745185 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1036/2090] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:36 2022] PUT /identity/v3/projects/e7987650d9ee4c74950fd33efd9f2514/users/615c6cdc808b489f80c7dc53c3ace941/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:36.756786 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cdc94-b154-4a2e-b8f8-499b78b44f43 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:36.757069 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cdc94-b154-4a2e-b8f8-499b78b44f43 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:36.757346 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cdc94-b154-4a2e-b8f8-499b78b44f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:36.767388 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-87ec4716-451a-4094-8754-c8e56b0d907e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b01fc37aab0c407d9ca6aeecff422159'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:36.768956 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-87ec4716-451a-4094-8754-c8e56b0d907e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b01fc37aab0c407d9ca6aeecff422159'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:36.770441 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-87ec4716-451a-4094-8754-c8e56b0d907e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b01fc37aab0c407d9ca6aeecff422159'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:36.792674 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-87ec4716-451a-4094-8754-c8e56b0d907e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b01fc37aab0c407d9ca6aeecff422159'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:36.800292 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-87ec4716-451a-4094-8754-c8e56b0d907e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b01fc37aab0c407d9ca6aeecff422159'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:36.814143 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-833cdc94-b154-4a2e-b8f8-499b78b44f43 None None] MFA Rules not processed for user `615c6cdc808b489f80c7dc53c3ace941`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:36.815386 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-833cdc94-b154-4a2e-b8f8-499b78b44f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:36.822028 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1055/2091] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:36 2022] DELETE /identity/v3/users/b01fc37aab0c407d9ca6aeecff422159 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:36.920038 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1037/2092] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:36 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:22:36.933154 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec4716-451a-4094-8754-c8e56b0d907e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:36.935085 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-87ec4716-451a-4094-8754-c8e56b0d907e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:37.020118 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec4716-451a-4094-8754-c8e56b0d907e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:37.024118 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc376a-ba1d-45d5-a2ae-2760ede0db2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:37.025560 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc376a-ba1d-45d5-a2ae-2760ede0db2f service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:37.026257 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc376a-ba1d-45d5-a2ae-2760ede0db2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:37.026558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc376a-ba1d-45d5-a2ae-2760ede0db2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:37.120898 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc376a-ba1d-45d5-a2ae-2760ede0db2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:37.122893 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc376a-ba1d-45d5-a2ae-2760ede0db2f service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:37.138012 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1056/2093] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:37.141214 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833cdc94-b154-4a2e-b8f8-499b78b44f43 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:37.145267 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-833cdc94-b154-4a2e-b8f8-499b78b44f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:37.255197 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:37.304789 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833cdc94-b154-4a2e-b8f8-499b78b44f43 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:37.385334 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a495b-858d-48e4-825c-07acc1e488b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:37.386037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a495b-858d-48e4-825c-07acc1e488b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:37.386719 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a495b-858d-48e4-825c-07acc1e488b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:37.386998 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a495b-858d-48e4-825c-07acc1e488b4 admin admin] PATH_INFO: `/v3/projects/b594579fd4534918acf5fd1e38cff36b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:37.388975 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761a495b-858d-48e4-825c-07acc1e488b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b594579fd4534918acf5fd1e38cff36b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:37.391423 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761a495b-858d-48e4-825c-07acc1e488b4 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:37.462365 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-761a495b-858d-48e4-825c-07acc1e488b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b594579fd4534918acf5fd1e38cff36b', 'request_id': 'req-761a495b-858d-48e4-825c-07acc1e488b4'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:37.468979 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-761a495b-858d-48e4-825c-07acc1e488b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b594579fd4534918acf5fd1e38cff36b', 'request_id': 'req-761a495b-858d-48e4-825c-07acc1e488b4'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:37.476300 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-761a495b-858d-48e4-825c-07acc1e488b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b594579fd4534918acf5fd1e38cff36b', 'request_id': 'req-761a495b-858d-48e4-825c-07acc1e488b4'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:37.495931 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-761a495b-858d-48e4-825c-07acc1e488b4 admin admin] The token cache is being invalidate because project b594579fd4534918acf5fd1e38cff36b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:37.496230 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-761a495b-858d-48e4-825c-07acc1e488b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:37.496985 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-761a495b-858d-48e4-825c-07acc1e488b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:37.498980 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1038/2094] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:37 2022] DELETE /identity/v3/projects/b594579fd4534918acf5fd1e38cff36b => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:37.504706 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d4cbf-5575-4e70-bafc-c8033dc12316 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:37.505195 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d4cbf-5575-4e70-bafc-c8033dc12316 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:37.505822 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d4cbf-5575-4e70-bafc-c8033dc12316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:37.527274 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-274d4cbf-5575-4e70-bafc-c8033dc12316 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:37.528694 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-274d4cbf-5575-4e70-bafc-c8033dc12316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:37.586138 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:37.619798 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1057/2095] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:37.621844 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359fca0-a28b-4d1e-938a-b894051e5dd6 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:37.622356 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359fca0-a28b-4d1e-938a-b894051e5dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:37.622537 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359fca0-a28b-4d1e-938a-b894051e5dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:37.643985 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-7359fca0-a28b-4d1e-938a-b894051e5dd6 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:37.644949 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-7359fca0-a28b-4d1e-938a-b894051e5dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:37.656443 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1039/2096] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:37.660297 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d4cbf-5575-4e70-bafc-c8033dc12316 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:37.670528 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d4cbf-5575-4e70-bafc-c8033dc12316 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:37.686532 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0804a09-a66b-41ce-a94d-f226e753335a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:37.687721 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0804a09-a66b-41ce-a94d-f226e753335a admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:37.688101 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0804a09-a66b-41ce-a94d-f226e753335a admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:37.688449 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0804a09-a66b-41ce-a94d-f226e753335a admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:37.689724 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0804a09-a66b-41ce-a94d-f226e753335a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:22:37.690284 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0804a09-a66b-41ce-a94d-f226e753335a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:37.692094 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0804a09-a66b-41ce-a94d-f226e753335a admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:37.705595 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1058/2097] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:22:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:22:37.710360 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7359fca0-a28b-4d1e-938a-b894051e5dd6 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:37.721909 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7359fca0-a28b-4d1e-938a-b894051e5dd6 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:37.727709 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9527cec9-7b40-484d-b135-7eba44e465b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:37.728341 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527cec9-7b40-484d-b135-7eba44e465b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:37.728823 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527cec9-7b40-484d-b135-7eba44e465b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:37.729105 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527cec9-7b40-484d-b135-7eba44e465b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:37.729799 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9527cec9-7b40-484d-b135-7eba44e465b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:37.731743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9527cec9-7b40-484d-b135-7eba44e465b1 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:37.770239 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0804a09-a66b-41ce-a94d-f226e753335a admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:37.770463 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1040/2098] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:37 2022] POST /identity/v3/projects => generated 359 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:37.782258 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0804a09-a66b-41ce-a94d-f226e753335a admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:37.793693 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9527cec9-7b40-484d-b135-7eba44e465b1 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:37.795837 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9527cec9-7b40-484d-b135-7eba44e465b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:37.847255 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:37.856541 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42af93a-f923-4b2e-a544-ce015783ef43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:37.857218 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42af93a-f923-4b2e-a544-ce015783ef43 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:37.857756 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42af93a-f923-4b2e-a544-ce015783ef43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:37.858068 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42af93a-f923-4b2e-a544-ce015783ef43 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:37.859298 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42af93a-f923-4b2e-a544-ce015783ef43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:37.873818 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42af93a-f923-4b2e-a544-ce015783ef43 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:37.874083 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9527cec9-7b40-484d-b135-7eba44e465b1 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:37.877865 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3bc718-fb08-41be-865d-e73f5a8a8071 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:37.878535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bc718-fb08-41be-865d-e73f5a8a8071 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:37.878855 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bc718-fb08-41be-865d-e73f5a8a8071 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:37.879138 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bc718-fb08-41be-865d-e73f5a8a8071 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:37.885704 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ed3bc718-fb08-41be-865d-e73f5a8a8071 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:37.886066 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1059/2099] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:37 2022] POST /identity/v3/users => generated 280 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:37.889822 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42af93a-f923-4b2e-a544-ce015783ef43 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:37.899215 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42af93a-f923-4b2e-a544-ce015783ef43 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:37.903936 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f5d71-d500-4acc-b357-b6d698658d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:37.904542 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f5d71-d500-4acc-b357-b6d698658d4d admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:37.904822 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f5d71-d500-4acc-b357-b6d698658d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:37.905113 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f5d71-d500-4acc-b357-b6d698658d4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:37.905790 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017f5d71-d500-4acc-b357-b6d698658d4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:37.907398 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017f5d71-d500-4acc-b357-b6d698658d4d admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:37.918081 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1060/2100] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:37.921907 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f5d71-d500-4acc-b357-b6d698658d4d admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:37.932354 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f5d71-d500-4acc-b357-b6d698658d4d admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:37.937480 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c83ba-7d0b-4b04-b843-3fd5bb363db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:37.938112 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c83ba-7d0b-4b04-b843-3fd5bb363db5 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:37.938403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c83ba-7d0b-4b04-b843-3fd5bb363db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:37.938679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c83ba-7d0b-4b04-b843-3fd5bb363db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:37.939365 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260c83ba-7d0b-4b04-b843-3fd5bb363db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:37.940968 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260c83ba-7d0b-4b04-b843-3fd5bb363db5 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:37.947372 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3bc718-fb08-41be-865d-e73f5a8a8071 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:37.948808 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3bc718-fb08-41be-865d-e73f5a8a8071 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:37.952018 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1061/2101] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:37.956064 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c83ba-7d0b-4b04-b843-3fd5bb363db5 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:37.958139 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1041/2102] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:37.965718 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c83ba-7d0b-4b04-b843-3fd5bb363db5 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:37.970549 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3208b-41c3-4605-9ee5-d0a3fd76b70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:37.971210 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3208b-41c3-4605-9ee5-d0a3fd76b70c admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:37.971511 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3208b-41c3-4605-9ee5-d0a3fd76b70c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:37.971810 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3208b-41c3-4605-9ee5-d0a3fd76b70c admin admin] PATH_INFO: `/v3/projects/9484881ef239402688cc44222e9dd9c1/users/57911857a45e402f9614f3a5a1decc31/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:37.986691 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e3208b-41c3-4605-9ee5-d0a3fd76b70c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9484881ef239402688cc44222e9dd9c1, user_id=57911857a45e402f9614f3a5a1decc31, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:37.989246 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e3208b-41c3-4605-9ee5-d0a3fd76b70c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:38.004188 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1062/2103] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:37 2022] PUT /identity/v3/projects/9484881ef239402688cc44222e9dd9c1/users/57911857a45e402f9614f3a5a1decc31/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:22:38.019273 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f19beb-027d-4c5e-8340-67b6bb6c0bbe None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:38.019642 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f19beb-027d-4c5e-8340-67b6bb6c0bbe None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:38.020008 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f19beb-027d-4c5e-8340-67b6bb6c0bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:38.048481 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-22f19beb-027d-4c5e-8340-67b6bb6c0bbe None None] MFA Rules not processed for user `57911857a45e402f9614f3a5a1decc31`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:38.049815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-22f19beb-027d-4c5e-8340-67b6bb6c0bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:38.119384 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1042/2104] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:38 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:22:38.154162 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3208b-41c3-4605-9ee5-d0a3fd76b70c admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:38.155933 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f0e3208b-41c3-4605-9ee5-d0a3fd76b70c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:38.229388 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3208b-41c3-4605-9ee5-d0a3fd76b70c admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:38.234551 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e168fd-ae15-4f79-9eb9-f71bdfd0caaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:38.235279 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e168fd-ae15-4f79-9eb9-f71bdfd0caaf service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:38.235763 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e168fd-ae15-4f79-9eb9-f71bdfd0caaf service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:38.236286 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e168fd-ae15-4f79-9eb9-f71bdfd0caaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:38.251058 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e168fd-ae15-4f79-9eb9-f71bdfd0caaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:38.252594 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e168fd-ae15-4f79-9eb9-f71bdfd0caaf service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:38.272161 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1063/2105] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:38 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:38.286758 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f19beb-027d-4c5e-8340-67b6bb6c0bbe None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:38.352418 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f19beb-027d-4c5e-8340-67b6bb6c0bbe None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:38.357666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d46c4-ff0d-4764-9391-862ea39d2dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:38.358343 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d46c4-ff0d-4764-9391-862ea39d2dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:38.358631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d46c4-ff0d-4764-9391-862ea39d2dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:38.358909 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d46c4-ff0d-4764-9391-862ea39d2dcb admin admin] PATH_INFO: `/v3/users/57911857a45e402f9614f3a5a1decc31` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:38.360117 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374d46c4-ff0d-4764-9391-862ea39d2dcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=57911857a45e402f9614f3a5a1decc31)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:38.362081 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374d46c4-ff0d-4764-9391-862ea39d2dcb admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:38.389287 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:38.408883 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-374d46c4-ff0d-4764-9391-862ea39d2dcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57911857a45e402f9614f3a5a1decc31'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:38.409532 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-374d46c4-ff0d-4764-9391-862ea39d2dcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57911857a45e402f9614f3a5a1decc31'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:38.418509 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-374d46c4-ff0d-4764-9391-862ea39d2dcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57911857a45e402f9614f3a5a1decc31'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:38.430304 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-374d46c4-ff0d-4764-9391-862ea39d2dcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57911857a45e402f9614f3a5a1decc31'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:38.437174 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-374d46c4-ff0d-4764-9391-862ea39d2dcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57911857a45e402f9614f3a5a1decc31'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:38.443079 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1043/2106] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:38 2022] DELETE /identity/v3/users/57911857a45e402f9614f3a5a1decc31 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:38.448440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d5389-6210-4ffe-8735-d8a9de02263e None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:38.448732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d5389-6210-4ffe-8735-d8a9de02263e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:38.449012 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d5389-6210-4ffe-8735-d8a9de02263e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:38.475880 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-294d5389-6210-4ffe-8735-d8a9de02263e None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:38.477183 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-294d5389-6210-4ffe-8735-d8a9de02263e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:38.495756 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:38.543071 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1064/2107] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:38.553639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d46c4-ff0d-4764-9391-862ea39d2dcb admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:38.555066 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-374d46c4-ff0d-4764-9391-862ea39d2dcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:38.621941 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d46c4-ff0d-4764-9391-862ea39d2dcb admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:38.625533 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446540f0-9ac7-402c-b4a2-5fb4ac74e404 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:38.626178 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446540f0-9ac7-402c-b4a2-5fb4ac74e404 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:38.626458 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446540f0-9ac7-402c-b4a2-5fb4ac74e404 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:38.626733 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446540f0-9ac7-402c-b4a2-5fb4ac74e404 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:38.636063 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446540f0-9ac7-402c-b4a2-5fb4ac74e404 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:38.637491 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446540f0-9ac7-402c-b4a2-5fb4ac74e404 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:38.646818 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1044/2108] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:39.156222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d5389-6210-4ffe-8735-d8a9de02263e None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:39.158020 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-294d5389-6210-4ffe-8735-d8a9de02263e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:39.170740 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d5389-6210-4ffe-8735-d8a9de02263e None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:39.175329 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a018eb-3af7-4edc-9747-7da3e50435d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:39.175982 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a018eb-3af7-4edc-9747-7da3e50435d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:39.176284 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a018eb-3af7-4edc-9747-7da3e50435d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:39.176542 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a018eb-3af7-4edc-9747-7da3e50435d9 admin admin] PATH_INFO: `/v3/projects/9484881ef239402688cc44222e9dd9c1` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:39.177705 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a018eb-3af7-4edc-9747-7da3e50435d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9484881ef239402688cc44222e9dd9c1)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:39.179648 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a018eb-3af7-4edc-9747-7da3e50435d9 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:39.226646 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a8a018eb-3af7-4edc-9747-7da3e50435d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9484881ef239402688cc44222e9dd9c1', 'request_id': 'req-a8a018eb-3af7-4edc-9747-7da3e50435d9'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:39.233539 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a8a018eb-3af7-4edc-9747-7da3e50435d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9484881ef239402688cc44222e9dd9c1', 'request_id': 'req-a8a018eb-3af7-4edc-9747-7da3e50435d9'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:39.239634 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a8a018eb-3af7-4edc-9747-7da3e50435d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9484881ef239402688cc44222e9dd9c1', 'request_id': 'req-a8a018eb-3af7-4edc-9747-7da3e50435d9'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:39.256701 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a8a018eb-3af7-4edc-9747-7da3e50435d9 admin admin] The token cache is being invalidate because project 9484881ef239402688cc44222e9dd9c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:39.257220 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a8a018eb-3af7-4edc-9747-7da3e50435d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:39.257955 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a8a018eb-3af7-4edc-9747-7da3e50435d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:39.260140 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1065/2109] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:39 2022] DELETE /identity/v3/projects/9484881ef239402688cc44222e9dd9c1 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:39.267002 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b9145-2903-47e0-963a-7c7bc85571e2 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:39.267562 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b9145-2903-47e0-963a-7c7bc85571e2 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:39.267847 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b9145-2903-47e0-963a-7c7bc85571e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:39.289785 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-1b7b9145-2903-47e0-963a-7c7bc85571e2 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:39.291516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-1b7b9145-2903-47e0-963a-7c7bc85571e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:39.353833 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1045/2110] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:39.362270 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bfdf4a-acde-4f6c-a649-fc3bc35910bf None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:39.362587 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bfdf4a-acde-4f6c-a649-fc3bc35910bf None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:39.362866 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bfdf4a-acde-4f6c-a649-fc3bc35910bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:39.382545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-07bfdf4a-acde-4f6c-a649-fc3bc35910bf None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:39.383844 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-07bfdf4a-acde-4f6c-a649-fc3bc35910bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:39.392235 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1066/2111] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:39.396749 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b9145-2903-47e0-963a-7c7bc85571e2 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:39.406573 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b9145-2903-47e0-963a-7c7bc85571e2 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:39.411046 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68c766-b68f-4c9d-b592-97d2ddfff8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:39.411647 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68c766-b68f-4c9d-b592-97d2ddfff8aa admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:39.411926 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68c766-b68f-4c9d-b592-97d2ddfff8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:39.412196 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68c766-b68f-4c9d-b592-97d2ddfff8aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:39.412888 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68c766-b68f-4c9d-b592-97d2ddfff8aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:22:39.413187 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68c766-b68f-4c9d-b592-97d2ddfff8aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:39.414820 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68c766-b68f-4c9d-b592-97d2ddfff8aa admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:39.426550 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1046/2112] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:22:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:22:39.430714 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bfdf4a-acde-4f6c-a649-fc3bc35910bf None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:39.440448 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bfdf4a-acde-4f6c-a649-fc3bc35910bf None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:39.445497 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2cb460-e4df-4e32-a7ba-025e87a58491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:39.446140 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2cb460-e4df-4e32-a7ba-025e87a58491 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:39.446506 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2cb460-e4df-4e32-a7ba-025e87a58491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:39.446899 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2cb460-e4df-4e32-a7ba-025e87a58491 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:39.447573 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2cb460-e4df-4e32-a7ba-025e87a58491 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:39.449451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2cb460-e4df-4e32-a7ba-025e87a58491 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:39.475723 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1067/2113] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:39 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:39.480561 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68c766-b68f-4c9d-b592-97d2ddfff8aa admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:39.490556 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68c766-b68f-4c9d-b592-97d2ddfff8aa admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:39.519945 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:39.547907 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f2c68-38ee-4a8b-ad3d-548c9de4a627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:39.548462 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f2c68-38ee-4a8b-ad3d-548c9de4a627 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:39.548746 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f2c68-38ee-4a8b-ad3d-548c9de4a627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:39.549023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f2c68-38ee-4a8b-ad3d-548c9de4a627 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:39.549687 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67f2c68-38ee-4a8b-ad3d-548c9de4a627 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:39.551644 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67f2c68-38ee-4a8b-ad3d-548c9de4a627 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:39.572785 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1047/2114] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:39 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:39.576879 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2cb460-e4df-4e32-a7ba-025e87a58491 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:39.588157 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2cb460-e4df-4e32-a7ba-025e87a58491 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:39.593002 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9080911-bcae-483c-b2fc-1b626c0b206b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:39.593633 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9080911-bcae-483c-b2fc-1b626c0b206b admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:39.593901 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9080911-bcae-483c-b2fc-1b626c0b206b admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:39.594219 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9080911-bcae-483c-b2fc-1b626c0b206b admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:39.594919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9080911-bcae-483c-b2fc-1b626c0b206b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:39.596512 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9080911-bcae-483c-b2fc-1b626c0b206b admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:39.610622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1068/2115] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:39.611941 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f2c68-38ee-4a8b-ad3d-548c9de4a627 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:39.622726 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f2c68-38ee-4a8b-ad3d-548c9de4a627 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:39.627452 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfac872-2c48-467d-b0bd-0f0e1042d7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:39.628064 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfac872-2c48-467d-b0bd-0f0e1042d7b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:39.628342 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfac872-2c48-467d-b0bd-0f0e1042d7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:39.628614 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfac872-2c48-467d-b0bd-0f0e1042d7b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:39.629311 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfac872-2c48-467d-b0bd-0f0e1042d7b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:39.630923 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfac872-2c48-467d-b0bd-0f0e1042d7b3 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:39.641654 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1048/2116] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:39.646138 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9080911-bcae-483c-b2fc-1b626c0b206b admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:39.656309 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9080911-bcae-483c-b2fc-1b626c0b206b admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:39.661212 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19bf352-73c9-4e29-bed2-2feab04c2529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:39.661938 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bf352-73c9-4e29-bed2-2feab04c2529 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:39.662248 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bf352-73c9-4e29-bed2-2feab04c2529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:39.662522 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bf352-73c9-4e29-bed2-2feab04c2529 admin admin] PATH_INFO: `/v3/projects/04ba41a28b1d4155a03f67e97956e6b0/users/602b5b67f122461080901c10f626791c/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:39.677583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19bf352-73c9-4e29-bed2-2feab04c2529 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04ba41a28b1d4155a03f67e97956e6b0, user_id=602b5b67f122461080901c10f626791c, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:39.680256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19bf352-73c9-4e29-bed2-2feab04c2529 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:39.693582 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1069/2117] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:39 2022] PUT /identity/v3/projects/04ba41a28b1d4155a03f67e97956e6b0/users/602b5b67f122461080901c10f626791c/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:39.707456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59424571-e0df-4afb-b8fe-adabb906a5eb None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:39.707744 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59424571-e0df-4afb-b8fe-adabb906a5eb None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:39.708058 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59424571-e0df-4afb-b8fe-adabb906a5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:39.740222 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-59424571-e0df-4afb-b8fe-adabb906a5eb None None] MFA Rules not processed for user `602b5b67f122461080901c10f626791c`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:39.741588 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-59424571-e0df-4afb-b8fe-adabb906a5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:39.813619 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1049/2118] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:39.818845 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19bf352-73c9-4e29-bed2-2feab04c2529 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:39.867397 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:39.887770 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19bf352-73c9-4e29-bed2-2feab04c2529 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:39.892634 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe1ea9-470c-4aec-a532-ec9e394a348d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:39.893266 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe1ea9-470c-4aec-a532-ec9e394a348d admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:39.893550 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe1ea9-470c-4aec-a532-ec9e394a348d admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:39.893826 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe1ea9-470c-4aec-a532-ec9e394a348d admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:39.894533 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbe1ea9-470c-4aec-a532-ec9e394a348d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:39.896449 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbe1ea9-470c-4aec-a532-ec9e394a348d admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:39.935935 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1070/2119] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:39 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:39.937197 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59424571-e0df-4afb-b8fe-adabb906a5eb None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:39.949025 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59424571-e0df-4afb-b8fe-adabb906a5eb None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:40.008746 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09625e4-e193-42ae-bd84-d3d1941931db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:40.009499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09625e4-e193-42ae-bd84-d3d1941931db admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:40.009777 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09625e4-e193-42ae-bd84-d3d1941931db admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:40.010085 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09625e4-e193-42ae-bd84-d3d1941931db admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:40.010762 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09625e4-e193-42ae-bd84-d3d1941931db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:40.012680 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09625e4-e193-42ae-bd84-d3d1941931db admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:40.035049 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1050/2120] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:39 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:40.039185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe1ea9-470c-4aec-a532-ec9e394a348d admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:40.050990 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe1ea9-470c-4aec-a532-ec9e394a348d admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:40.056374 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55e320-5d52-4f3c-8b6f-ca6c7097ee6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:40.057040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55e320-5d52-4f3c-8b6f-ca6c7097ee6a admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:40.057319 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55e320-5d52-4f3c-8b6f-ca6c7097ee6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:40.057593 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55e320-5d52-4f3c-8b6f-ca6c7097ee6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:40.058334 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc55e320-5d52-4f3c-8b6f-ca6c7097ee6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:40.059927 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc55e320-5d52-4f3c-8b6f-ca6c7097ee6a admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:40.070949 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1071/2121] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:40.076197 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09625e4-e193-42ae-bd84-d3d1941931db admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:40.086865 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09625e4-e193-42ae-bd84-d3d1941931db admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:40.092021 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fbaa70-728a-451a-a46f-1fe3058c1f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:40.092695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fbaa70-728a-451a-a46f-1fe3058c1f01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:40.092972 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fbaa70-728a-451a-a46f-1fe3058c1f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:40.093244 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fbaa70-728a-451a-a46f-1fe3058c1f01 admin admin] PATH_INFO: `/v3/projects/4360b68c2800498e9eb31bc19e54d5e2/users/dab06984b1d342e1996325ee6c8e4185/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:40.113876 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fbaa70-728a-451a-a46f-1fe3058c1f01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4360b68c2800498e9eb31bc19e54d5e2, user_id=dab06984b1d342e1996325ee6c8e4185, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:40.116560 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fbaa70-728a-451a-a46f-1fe3058c1f01 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:40.130488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1051/2122] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:40 2022] PUT /identity/v3/projects/4360b68c2800498e9eb31bc19e54d5e2/users/dab06984b1d342e1996325ee6c8e4185/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:40.135124 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55e320-5d52-4f3c-8b6f-ca6c7097ee6a admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:40.145227 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55e320-5d52-4f3c-8b6f-ca6c7097ee6a admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:40.209617 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b2893-66b6-4055-8464-806a977f7368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:40.210455 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b2893-66b6-4055-8464-806a977f7368 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:40.210733 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b2893-66b6-4055-8464-806a977f7368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:40.211006 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b2893-66b6-4055-8464-806a977f7368 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:40.211733 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b2893-66b6-4055-8464-806a977f7368 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:40.213354 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b2893-66b6-4055-8464-806a977f7368 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:40.227158 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1072/2123] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:40.228919 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fbaa70-728a-451a-a46f-1fe3058c1f01 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:40.240022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fbaa70-728a-451a-a46f-1fe3058c1f01 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:40.245654 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f21dda-4df7-47b8-a017-376a5db5f1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:40.246359 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f21dda-4df7-47b8-a017-376a5db5f1b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:40.246639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f21dda-4df7-47b8-a017-376a5db5f1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:40.246911 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f21dda-4df7-47b8-a017-376a5db5f1b3 admin admin] PATH_INFO: `/v3/projects/4360b68c2800498e9eb31bc19e54d5e2/users/dab06984b1d342e1996325ee6c8e4185/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:40.249568 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f21dda-4df7-47b8-a017-376a5db5f1b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4360b68c2800498e9eb31bc19e54d5e2, user_id=dab06984b1d342e1996325ee6c8e4185, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:40.253386 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f21dda-4df7-47b8-a017-376a5db5f1b3 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:40.256606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:40.269511 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1052/2124] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:40 2022] PUT /identity/v3/projects/4360b68c2800498e9eb31bc19e54d5e2/users/dab06984b1d342e1996325ee6c8e4185/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:22:40.280139 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b56b3-cc67-476c-860b-16d0e9012abd None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:40.280440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b56b3-cc67-476c-860b-16d0e9012abd None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:40.280740 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b56b3-cc67-476c-860b-16d0e9012abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:40.311827 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-246b56b3-cc67-476c-860b-16d0e9012abd None None] MFA Rules not processed for user `dab06984b1d342e1996325ee6c8e4185`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:40.313193 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-246b56b3-cc67-476c-860b-16d0e9012abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:40.449642 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1073/2125] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:40 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:40.506885 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f21dda-4df7-47b8-a017-376a5db5f1b3 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:40.511566 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b8f21dda-4df7-47b8-a017-376a5db5f1b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:40.583455 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f21dda-4df7-47b8-a017-376a5db5f1b3 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:40.588350 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c3ac6-f858-4085-a13e-93ed2f2e1309 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:40.589002 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c3ac6-f858-4085-a13e-93ed2f2e1309 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:40.589254 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c3ac6-f858-4085-a13e-93ed2f2e1309 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:40.589527 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c3ac6-f858-4085-a13e-93ed2f2e1309 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:40.673433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c3ac6-f858-4085-a13e-93ed2f2e1309 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:40.675038 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c3ac6-f858-4085-a13e-93ed2f2e1309 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:40.688064 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1053/2126] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:44.018989 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b56b3-cc67-476c-860b-16d0e9012abd None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:44.020690 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-246b56b3-cc67-476c-860b-16d0e9012abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:44.089085 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:44.124487 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b56b3-cc67-476c-860b-16d0e9012abd None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:44.131606 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769a13e-6c01-4bfa-9031-e34254894aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:44.132293 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769a13e-6c01-4bfa-9031-e34254894aa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:44.132574 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769a13e-6c01-4bfa-9031-e34254894aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:44.132849 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769a13e-6c01-4bfa-9031-e34254894aa6 admin admin] PATH_INFO: `/v3/users/402e73b674d44334b83ce2e8d4969a10` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:44.134125 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769a13e-6c01-4bfa-9031-e34254894aa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=402e73b674d44334b83ce2e8d4969a10)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:44.136833 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769a13e-6c01-4bfa-9031-e34254894aa6 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:44.242058 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-5769a13e-6c01-4bfa-9031-e34254894aa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '402e73b674d44334b83ce2e8d4969a10'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:44.243079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-5769a13e-6c01-4bfa-9031-e34254894aa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '402e73b674d44334b83ce2e8d4969a10'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:44.243778 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-5769a13e-6c01-4bfa-9031-e34254894aa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '402e73b674d44334b83ce2e8d4969a10'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:44.269001 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-5769a13e-6c01-4bfa-9031-e34254894aa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '402e73b674d44334b83ce2e8d4969a10'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:44.286579 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-5769a13e-6c01-4bfa-9031-e34254894aa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '402e73b674d44334b83ce2e8d4969a10'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:44.312402 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1074/2127] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:44 2022] DELETE /identity/v3/users/402e73b674d44334b83ce2e8d4969a10 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:44.316984 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8299cb-fe07-40e8-871b-e6e6699ce510 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:44.317452 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8299cb-fe07-40e8-871b-e6e6699ce510 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:44.317668 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8299cb-fe07-40e8-871b-e6e6699ce510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:44.340466 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-9a8299cb-fe07-40e8-871b-e6e6699ce510 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:44.341756 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9a8299cb-fe07-40e8-871b-e6e6699ce510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:44.427743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1054/2128] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:22:44.438751 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769a13e-6c01-4bfa-9031-e34254894aa6 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:44.440187 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5769a13e-6c01-4bfa-9031-e34254894aa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:44.527608 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769a13e-6c01-4bfa-9031-e34254894aa6 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:44.531339 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f1d67-3aea-4178-8fe9-e4237ced54d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:44.531967 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f1d67-3aea-4178-8fe9-e4237ced54d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:44.532935 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f1d67-3aea-4178-8fe9-e4237ced54d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:44.532935 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f1d67-3aea-4178-8fe9-e4237ced54d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:44.541930 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f1d67-3aea-4178-8fe9-e4237ced54d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:44.543468 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f1d67-3aea-4178-8fe9-e4237ced54d8 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:44.556195 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1075/2129] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:44.952819 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8299cb-fe07-40e8-871b-e6e6699ce510 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:44.954343 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9a8299cb-fe07-40e8-871b-e6e6699ce510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:44.998781 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8299cb-fe07-40e8-871b-e6e6699ce510 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:45.006692 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06deea7-4a74-4523-817d-b796cc0da6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:45.008119 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06deea7-4a74-4523-817d-b796cc0da6c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:45.008614 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06deea7-4a74-4523-817d-b796cc0da6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:45.008873 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06deea7-4a74-4523-817d-b796cc0da6c0 admin admin] PATH_INFO: `/v3/projects/92d3a03f469746a1b8f379f2a61069ea` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:45.010316 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06deea7-4a74-4523-817d-b796cc0da6c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92d3a03f469746a1b8f379f2a61069ea)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:45.012767 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06deea7-4a74-4523-817d-b796cc0da6c0 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:45.110683 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:INSERT=1,keystone:SELECT=96,keystone:DELETE=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:45.115269 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f06deea7-4a74-4523-817d-b796cc0da6c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92d3a03f469746a1b8f379f2a61069ea', 'request_id': 'req-f06deea7-4a74-4523-817d-b796cc0da6c0'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:45.124659 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f06deea7-4a74-4523-817d-b796cc0da6c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92d3a03f469746a1b8f379f2a61069ea', 'request_id': 'req-f06deea7-4a74-4523-817d-b796cc0da6c0'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:45.136672 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f06deea7-4a74-4523-817d-b796cc0da6c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92d3a03f469746a1b8f379f2a61069ea', 'request_id': 'req-f06deea7-4a74-4523-817d-b796cc0da6c0'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:45.167961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f06deea7-4a74-4523-817d-b796cc0da6c0 admin admin] The token cache is being invalidate because project 92d3a03f469746a1b8f379f2a61069ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:45.168693 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f06deea7-4a74-4523-817d-b796cc0da6c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:45.171097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f06deea7-4a74-4523-817d-b796cc0da6c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:45.173699 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1055/2130] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:44 2022] DELETE /identity/v3/projects/92d3a03f469746a1b8f379f2a61069ea => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:45.177667 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f1d67-3aea-4178-8fe9-e4237ced54d8 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:45.189931 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f1d67-3aea-4178-8fe9-e4237ced54d8 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:45.267871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a2235-1922-43a5-9442-ef0c597140c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:45.269396 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a2235-1922-43a5-9442-ef0c597140c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:45.269781 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a2235-1922-43a5-9442-ef0c597140c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:45.270108 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a2235-1922-43a5-9442-ef0c597140c6 admin admin] PATH_INFO: `/v3/users/2b8caa5cdc31471fa7ad04c2fd0ab286` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:45.271838 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35a2235-1922-43a5-9442-ef0c597140c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b8caa5cdc31471fa7ad04c2fd0ab286)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:45.274152 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35a2235-1922-43a5-9442-ef0c597140c6 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:45.275889 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:45.338274 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a35a2235-1922-43a5-9442-ef0c597140c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b8caa5cdc31471fa7ad04c2fd0ab286'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:45.339392 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a35a2235-1922-43a5-9442-ef0c597140c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b8caa5cdc31471fa7ad04c2fd0ab286'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:45.340013 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a35a2235-1922-43a5-9442-ef0c597140c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b8caa5cdc31471fa7ad04c2fd0ab286'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:45.364221 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a35a2235-1922-43a5-9442-ef0c597140c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b8caa5cdc31471fa7ad04c2fd0ab286'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:45.372616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a35a2235-1922-43a5-9442-ef0c597140c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b8caa5cdc31471fa7ad04c2fd0ab286'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:45.387673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1076/2131] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:45 2022] DELETE /identity/v3/users/2b8caa5cdc31471fa7ad04c2fd0ab286 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:45.633207 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06deea7-4a74-4523-817d-b796cc0da6c0 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:45.634989 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f06deea7-4a74-4523-817d-b796cc0da6c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:45.687480 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06deea7-4a74-4523-817d-b796cc0da6c0 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:45.840558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf840fe7-ab97-4a26-9780-999f92402766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:45.842821 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf840fe7-ab97-4a26-9780-999f92402766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:45.842821 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf840fe7-ab97-4a26-9780-999f92402766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:45.843130 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf840fe7-ab97-4a26-9780-999f92402766 admin admin] PATH_INFO: `/v3/users/602b5b67f122461080901c10f626791c` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:45.847203 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf840fe7-ab97-4a26-9780-999f92402766 admin admin] RBAC: Authorizing `identity:delete_user(user_id=602b5b67f122461080901c10f626791c)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:45.850071 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf840fe7-ab97-4a26-9780-999f92402766 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:45.935922 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-bf840fe7-ab97-4a26-9780-999f92402766 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '602b5b67f122461080901c10f626791c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:45.936937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-bf840fe7-ab97-4a26-9780-999f92402766 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '602b5b67f122461080901c10f626791c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:45.951384 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-bf840fe7-ab97-4a26-9780-999f92402766 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '602b5b67f122461080901c10f626791c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:45.969534 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a2235-1922-43a5-9442-ef0c597140c6 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:45.971366 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a35a2235-1922-43a5-9442-ef0c597140c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:45.992548 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-bf840fe7-ab97-4a26-9780-999f92402766 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '602b5b67f122461080901c10f626791c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:45.994685 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a2235-1922-43a5-9442-ef0c597140c6 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:46.009872 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-bf840fe7-ab97-4a26-9780-999f92402766 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '602b5b67f122461080901c10f626791c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:46.018010 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1056/2132] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:45 2022] DELETE /identity/v3/users/602b5b67f122461080901c10f626791c => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:46.022546 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef54f24-1b93-415f-9482-0665861e4afb None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:46.023025 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef54f24-1b93-415f-9482-0665861e4afb None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:46.023395 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef54f24-1b93-415f-9482-0665861e4afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:46.030530 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2fb24c-7608-4f8a-a66b-080684c81d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:46.031212 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fb24c-7608-4f8a-a66b-080684c81d65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:46.031500 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fb24c-7608-4f8a-a66b-080684c81d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:46.031782 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fb24c-7608-4f8a-a66b-080684c81d65 admin admin] PATH_INFO: `/v3/projects/70e1ee59339844b98a4c4fe41127e330` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:46.036182 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2fb24c-7608-4f8a-a66b-080684c81d65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70e1ee59339844b98a4c4fe41127e330)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:46.044779 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2fb24c-7608-4f8a-a66b-080684c81d65 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:46.052137 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-0ef54f24-1b93-415f-9482-0665861e4afb None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:46.052807 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0ef54f24-1b93-415f-9482-0665861e4afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:46.121844 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:46.145871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-5a2fb24c-7608-4f8a-a66b-080684c81d65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70e1ee59339844b98a4c4fe41127e330', 'request_id': 'req-5a2fb24c-7608-4f8a-a66b-080684c81d65'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:46.154539 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1057/2133] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:46.154539 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef54f24-1b93-415f-9482-0665861e4afb None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:46.154539 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0ef54f24-1b93-415f-9482-0665861e4afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:46.155102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-5a2fb24c-7608-4f8a-a66b-080684c81d65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70e1ee59339844b98a4c4fe41127e330', 'request_id': 'req-5a2fb24c-7608-4f8a-a66b-080684c81d65'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:46.162533 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-5a2fb24c-7608-4f8a-a66b-080684c81d65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70e1ee59339844b98a4c4fe41127e330', 'request_id': 'req-5a2fb24c-7608-4f8a-a66b-080684c81d65'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:46.187809 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-5a2fb24c-7608-4f8a-a66b-080684c81d65 admin admin] The token cache is being invalidate because project 70e1ee59339844b98a4c4fe41127e330 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:46.189134 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-5a2fb24c-7608-4f8a-a66b-080684c81d65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:46.190623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-5a2fb24c-7608-4f8a-a66b-080684c81d65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:46.192943 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1077/2134] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:45 2022] DELETE /identity/v3/projects/70e1ee59339844b98a4c4fe41127e330 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:46.194357 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2fb24c-7608-4f8a-a66b-080684c81d65 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:46.196643 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5a2fb24c-7608-4f8a-a66b-080684c81d65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:46.263940 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef54f24-1b93-415f-9482-0665861e4afb None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:46.295425 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2fb24c-7608-4f8a-a66b-080684c81d65 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:46.300399 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113968c0-f3f9-4894-b12b-3af1f54ba268 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:46.301032 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113968c0-f3f9-4894-b12b-3af1f54ba268 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:46.301313 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113968c0-f3f9-4894-b12b-3af1f54ba268 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:46.301588 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113968c0-f3f9-4894-b12b-3af1f54ba268 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:46.303634 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-113968c0-f3f9-4894-b12b-3af1f54ba268 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:46.320764 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:46.379909 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700a478-0f50-41d6-bf21-0dd519cf70c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:46.380918 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700a478-0f50-41d6-bf21-0dd519cf70c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:46.380995 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700a478-0f50-41d6-bf21-0dd519cf70c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:46.381425 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700a478-0f50-41d6-bf21-0dd519cf70c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:46.388186 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-a700a478-0f50-41d6-bf21-0dd519cf70c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:46.400495 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113968c0-f3f9-4894-b12b-3af1f54ba268 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:46.402855 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113968c0-f3f9-4894-b12b-3af1f54ba268 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:46.434070 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1078/2135] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:46.435840 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60a08c-e35e-4f2d-92da-a2de8cf369f9 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:46.436445 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60a08c-e35e-4f2d-92da-a2de8cf369f9 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:46.437419 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60a08c-e35e-4f2d-92da-a2de8cf369f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:46.499352 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-8e60a08c-e35e-4f2d-92da-a2de8cf369f9 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:46.501732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8e60a08c-e35e-4f2d-92da-a2de8cf369f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:46.524308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1079/2136] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:46.526524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329d88f-5c5c-4262-bc4e-ce6561b922b4 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:46.527087 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329d88f-5c5c-4262-bc4e-ce6561b922b4 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:46.527542 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329d88f-5c5c-4262-bc4e-ce6561b922b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:46.561869 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-b329d88f-5c5c-4262-bc4e-ce6561b922b4 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:46.563395 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b329d88f-5c5c-4262-bc4e-ce6561b922b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:46.574995 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a700a478-0f50-41d6-bf21-0dd519cf70c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:46.576570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a700a478-0f50-41d6-bf21-0dd519cf70c4 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:46.580220 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1080/2137] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:46.582886 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b329d88f-5c5c-4262-bc4e-ce6561b922b4 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:46.587980 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1058/2138] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:46.601069 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b329d88f-5c5c-4262-bc4e-ce6561b922b4 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:46.607463 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfeb4fd-253a-4bb3-8e5c-dc4d3565a8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:46.608102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfeb4fd-253a-4bb3-8e5c-dc4d3565a8dd admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:46.608455 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfeb4fd-253a-4bb3-8e5c-dc4d3565a8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:46.608735 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfeb4fd-253a-4bb3-8e5c-dc4d3565a8dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:46.610018 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfeb4fd-253a-4bb3-8e5c-dc4d3565a8dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:22:46.610018 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfeb4fd-253a-4bb3-8e5c-dc4d3565a8dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:46.618238 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfeb4fd-253a-4bb3-8e5c-dc4d3565a8dd admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:46.638907 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700a478-0f50-41d6-bf21-0dd519cf70c4 service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:46.639463 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1081/2139] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:22:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:22:46.678896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700a478-0f50-41d6-bf21-0dd519cf70c4 service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:46.699259 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c462bd-e64c-45d2-a16c-a1a0b3e1f1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:46.701146 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c462bd-e64c-45d2-a16c-a1a0b3e1f1cb admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:46.701493 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfeb4fd-253a-4bb3-8e5c-dc4d3565a8dd admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:46.702102 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c462bd-e64c-45d2-a16c-a1a0b3e1f1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:46.702866 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c462bd-e64c-45d2-a16c-a1a0b3e1f1cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:46.703427 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8bfeb4fd-253a-4bb3-8e5c-dc4d3565a8dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:46.704414 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c462bd-e64c-45d2-a16c-a1a0b3e1f1cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:46.706588 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c462bd-e64c-45d2-a16c-a1a0b3e1f1cb admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:46.730101 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfeb4fd-253a-4bb3-8e5c-dc4d3565a8dd admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:46.744032 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ddaafd-ef39-421e-8e34-6cc2f7a226c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:46.745219 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ddaafd-ef39-421e-8e34-6cc2f7a226c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:46.745514 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ddaafd-ef39-421e-8e34-6cc2f7a226c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:46.745798 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ddaafd-ef39-421e-8e34-6cc2f7a226c1 admin admin] PATH_INFO: `/v3/users/2291aad176c345d6a6fd116af02c8156` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:46.747995 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ddaafd-ef39-421e-8e34-6cc2f7a226c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2291aad176c345d6a6fd116af02c8156)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:46.751258 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ddaafd-ef39-421e-8e34-6cc2f7a226c1 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:46.765119 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1059/2140] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:46 2022] POST /identity/v3/projects => generated 377 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:22:46.766100 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c462bd-e64c-45d2-a16c-a1a0b3e1f1cb admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:46.827466 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-60ddaafd-ef39-421e-8e34-6cc2f7a226c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2291aad176c345d6a6fd116af02c8156'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:46.828273 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-60ddaafd-ef39-421e-8e34-6cc2f7a226c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2291aad176c345d6a6fd116af02c8156'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:46.828951 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-60ddaafd-ef39-421e-8e34-6cc2f7a226c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2291aad176c345d6a6fd116af02c8156'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:46.843727 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-60ddaafd-ef39-421e-8e34-6cc2f7a226c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2291aad176c345d6a6fd116af02c8156'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:46.852170 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-60ddaafd-ef39-421e-8e34-6cc2f7a226c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2291aad176c345d6a6fd116af02c8156'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:46.859052 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:46.876202 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1082/2141] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:46 2022] DELETE /identity/v3/users/2291aad176c345d6a6fd116af02c8156 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:22:46.878564 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ddaafd-ef39-421e-8e34-6cc2f7a226c1 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:46.880701 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-60ddaafd-ef39-421e-8e34-6cc2f7a226c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:46.899096 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c462bd-e64c-45d2-a16c-a1a0b3e1f1cb admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:46.976741 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:46.982343 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ddaafd-ef39-421e-8e34-6cc2f7a226c1 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:46.982701 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6de9ba-2ddb-4fbd-b462-da254e57c8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:46.983688 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6de9ba-2ddb-4fbd-b462-da254e57c8a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:46.984115 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6de9ba-2ddb-4fbd-b462-da254e57c8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:46.984479 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6de9ba-2ddb-4fbd-b462-da254e57c8a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:46.985330 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6de9ba-2ddb-4fbd-b462-da254e57c8a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:46.988933 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6de9ba-2ddb-4fbd-b462-da254e57c8a0 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:46.989361 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857cdc9-434c-4c40-924a-21fff4647ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:46.990941 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857cdc9-434c-4c40-924a-21fff4647ca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:46.991810 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857cdc9-434c-4c40-924a-21fff4647ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:46.993946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857cdc9-434c-4c40-924a-21fff4647ca8 admin admin] PATH_INFO: `/v3/projects/04ba41a28b1d4155a03f67e97956e6b0` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:46.996625 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b857cdc9-434c-4c40-924a-21fff4647ca8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04ba41a28b1d4155a03f67e97956e6b0)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:46.999241 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b857cdc9-434c-4c40-924a-21fff4647ca8 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:47.019131 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1060/2142] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:46 2022] POST /identity/v3/users => generated 289 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:47.020743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd8bff-0033-482a-800b-26cef2c723db None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:47.021078 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd8bff-0033-482a-800b-26cef2c723db None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:47.021369 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd8bff-0033-482a-800b-26cef2c723db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:47.043547 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-02fd8bff-0033-482a-800b-26cef2c723db None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:47.044799 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-02fd8bff-0033-482a-800b-26cef2c723db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:47.053343 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1061/2143] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:47.054280 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd8bff-0033-482a-800b-26cef2c723db None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:47.057493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-02fd8bff-0033-482a-800b-26cef2c723db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:47.066354 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b857cdc9-434c-4c40-924a-21fff4647ca8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04ba41a28b1d4155a03f67e97956e6b0', 'request_id': 'req-b857cdc9-434c-4c40-924a-21fff4647ca8'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:47.072242 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd8bff-0033-482a-800b-26cef2c723db None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:47.076145 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b857cdc9-434c-4c40-924a-21fff4647ca8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04ba41a28b1d4155a03f67e97956e6b0', 'request_id': 'req-b857cdc9-434c-4c40-924a-21fff4647ca8'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:47.083416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b857cdc9-434c-4c40-924a-21fff4647ca8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04ba41a28b1d4155a03f67e97956e6b0', 'request_id': 'req-b857cdc9-434c-4c40-924a-21fff4647ca8'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:47.100167 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b857cdc9-434c-4c40-924a-21fff4647ca8 admin admin] The token cache is being invalidate because project 04ba41a28b1d4155a03f67e97956e6b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:47.100509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b857cdc9-434c-4c40-924a-21fff4647ca8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:47.101189 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b857cdc9-434c-4c40-924a-21fff4647ca8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:47.102801 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1083/2144] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:46 2022] DELETE /identity/v3/projects/04ba41a28b1d4155a03f67e97956e6b0 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:47.106540 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857cdc9-434c-4c40-924a-21fff4647ca8 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:47.108107 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b857cdc9-434c-4c40-924a-21fff4647ca8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:47.157680 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26640206-a5d3-41d1-a3cd-b0a68dedd823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:47.158741 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26640206-a5d3-41d1-a3cd-b0a68dedd823 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:47.158741 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26640206-a5d3-41d1-a3cd-b0a68dedd823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:47.158916 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26640206-a5d3-41d1-a3cd-b0a68dedd823 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:47.159602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26640206-a5d3-41d1-a3cd-b0a68dedd823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:47.161239 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26640206-a5d3-41d1-a3cd-b0a68dedd823 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:47.174748 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1062/2145] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:47 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:47.175894 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26640206-a5d3-41d1-a3cd-b0a68dedd823 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:47.179494 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-26640206-a5d3-41d1-a3cd-b0a68dedd823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:47.183337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857cdc9-434c-4c40-924a-21fff4647ca8 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:47.188225 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785320e-4e3a-4c00-a775-5e634a8a7b1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:47.188929 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785320e-4e3a-4c00-a775-5e634a8a7b1f service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:47.189216 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785320e-4e3a-4c00-a775-5e634a8a7b1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:47.189498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785320e-4e3a-4c00-a775-5e634a8a7b1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:47.191363 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e785320e-4e3a-4c00-a775-5e634a8a7b1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:47.194466 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26640206-a5d3-41d1-a3cd-b0a68dedd823 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:47.200561 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e6402d-f67c-41a1-9c4c-3dcd2c9c15c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:47.201423 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6402d-f67c-41a1-9c4c-3dcd2c9c15c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:47.201603 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6402d-f67c-41a1-9c4c-3dcd2c9c15c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:47.201887 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6402d-f67c-41a1-9c4c-3dcd2c9c15c0 admin admin] PATH_INFO: `/v3/users/dab06984b1d342e1996325ee6c8e4185` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:47.203552 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e6402d-f67c-41a1-9c4c-3dcd2c9c15c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dab06984b1d342e1996325ee6c8e4185)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:47.205818 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e6402d-f67c-41a1-9c4c-3dcd2c9c15c0 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:47.206125 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785320e-4e3a-4c00-a775-5e634a8a7b1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:47.208176 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785320e-4e3a-4c00-a775-5e634a8a7b1f service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:47.221069 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1084/2146] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:47.223097 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785320e-4e3a-4c00-a775-5e634a8a7b1f service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:47.224593 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e785320e-4e3a-4c00-a775-5e634a8a7b1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:47.240291 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785320e-4e3a-4c00-a775-5e634a8a7b1f service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:47.246943 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cca450-95fa-411a-8f1e-0d6b08f7b4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:47.247628 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cca450-95fa-411a-8f1e-0d6b08f7b4cb admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:47.247938 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cca450-95fa-411a-8f1e-0d6b08f7b4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:47.248604 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cca450-95fa-411a-8f1e-0d6b08f7b4cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:47.249222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cca450-95fa-411a-8f1e-0d6b08f7b4cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:47.251146 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cca450-95fa-411a-8f1e-0d6b08f7b4cb admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:47.264409 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1085/2147] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:47 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:47.270022 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cca450-95fa-411a-8f1e-0d6b08f7b4cb admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:47.286121 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cca450-95fa-411a-8f1e-0d6b08f7b4cb admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:47.289854 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-93e6402d-f67c-41a1-9c4c-3dcd2c9c15c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dab06984b1d342e1996325ee6c8e4185'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:47.291174 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-93e6402d-f67c-41a1-9c4c-3dcd2c9c15c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dab06984b1d342e1996325ee6c8e4185'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:47.295577 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2f618-8b2d-4cb3-9181-06497bb2b974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:47.295577 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2f618-8b2d-4cb3-9181-06497bb2b974 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:47.295577 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2f618-8b2d-4cb3-9181-06497bb2b974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:47.295577 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2f618-8b2d-4cb3-9181-06497bb2b974 admin admin] PATH_INFO: `/v3/projects/7ed1fe0e6b2847e49cbbf7bc1cdfb8f4/users/f694fe71084c4651813a9372bb57f928/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:47.310675 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-93e6402d-f67c-41a1-9c4c-3dcd2c9c15c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dab06984b1d342e1996325ee6c8e4185'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:47.315752 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f2f618-8b2d-4cb3-9181-06497bb2b974 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ed1fe0e6b2847e49cbbf7bc1cdfb8f4, user_id=f694fe71084c4651813a9372bb57f928, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:47.321046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f2f618-8b2d-4cb3-9181-06497bb2b974 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:47.331781 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:47.354099 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1086/2148] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:47 2022] PUT /identity/v3/projects/7ed1fe0e6b2847e49cbbf7bc1cdfb8f4/users/f694fe71084c4651813a9372bb57f928/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:47.357282 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-93e6402d-f67c-41a1-9c4c-3dcd2c9c15c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dab06984b1d342e1996325ee6c8e4185'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:47.365831 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02866dd2-de90-4ac1-ae8b-58e1ba34d51b None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:47.369337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02866dd2-de90-4ac1-ae8b-58e1ba34d51b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:47.369782 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02866dd2-de90-4ac1-ae8b-58e1ba34d51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:47.370836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-93e6402d-f67c-41a1-9c4c-3dcd2c9c15c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dab06984b1d342e1996325ee6c8e4185'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:47.388293 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1063/2149] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:47 2022] DELETE /identity/v3/users/dab06984b1d342e1996325ee6c8e4185 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:47.444272 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-02866dd2-de90-4ac1-ae8b-58e1ba34d51b None None] MFA Rules not processed for user `f694fe71084c4651813a9372bb57f928`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:47.445818 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-02866dd2-de90-4ac1-ae8b-58e1ba34d51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:47.583084 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:47.620196 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1087/2150] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:47.621628 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e6402d-f67c-41a1-9c4c-3dcd2c9c15c0 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:47.624086 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-93e6402d-f67c-41a1-9c4c-3dcd2c9c15c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:47.750343 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e6402d-f67c-41a1-9c4c-3dcd2c9c15c0 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:47.776035 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d7030-8aad-4945-aef7-5da96ecda291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:47.776836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d7030-8aad-4945-aef7-5da96ecda291 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:47.777120 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d7030-8aad-4945-aef7-5da96ecda291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:47.777711 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d7030-8aad-4945-aef7-5da96ecda291 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:47.778431 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d7030-8aad-4945-aef7-5da96ecda291 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:47.781005 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d7030-8aad-4945-aef7-5da96ecda291 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:47.800236 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02866dd2-de90-4ac1-ae8b-58e1ba34d51b None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:47.800236 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-02866dd2-de90-4ac1-ae8b-58e1ba34d51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:47.842015 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02866dd2-de90-4ac1-ae8b-58e1ba34d51b None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:47.846651 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66cf09-0d7a-467f-9fc0-ca8a62a959d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:47.847302 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66cf09-0d7a-467f-9fc0-ca8a62a959d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:47.847588 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66cf09-0d7a-467f-9fc0-ca8a62a959d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:47.847871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66cf09-0d7a-467f-9fc0-ca8a62a959d4 admin admin] PATH_INFO: `/v3/projects/cc6cf109e9154af5a7e85431ee5e5c31` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:47.851053 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1064/2151] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:47 2022] POST /identity/v3/projects => generated 379 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:47.851196 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe66cf09-0d7a-467f-9fc0-ca8a62a959d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc6cf109e9154af5a7e85431ee5e5c31)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:47.851196 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe66cf09-0d7a-467f-9fc0-ca8a62a959d4 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:47.852092 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d7030-8aad-4945-aef7-5da96ecda291 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:47.923286 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fe66cf09-0d7a-467f-9fc0-ca8a62a959d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc6cf109e9154af5a7e85431ee5e5c31', 'request_id': 'req-fe66cf09-0d7a-467f-9fc0-ca8a62a959d4'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:47.934258 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fe66cf09-0d7a-467f-9fc0-ca8a62a959d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc6cf109e9154af5a7e85431ee5e5c31', 'request_id': 'req-fe66cf09-0d7a-467f-9fc0-ca8a62a959d4'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:47.948618 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fe66cf09-0d7a-467f-9fc0-ca8a62a959d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc6cf109e9154af5a7e85431ee5e5c31', 'request_id': 'req-fe66cf09-0d7a-467f-9fc0-ca8a62a959d4'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:47.950425 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d7030-8aad-4945-aef7-5da96ecda291 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:47.977371 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fe66cf09-0d7a-467f-9fc0-ca8a62a959d4 admin admin] The token cache is being invalidate because project cc6cf109e9154af5a7e85431ee5e5c31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:47.977486 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fe66cf09-0d7a-467f-9fc0-ca8a62a959d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:47.978858 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fe66cf09-0d7a-467f-9fc0-ca8a62a959d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:47.981055 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1088/2152] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:47 2022] DELETE /identity/v3/projects/cc6cf109e9154af5a7e85431ee5e5c31 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:22:47.981678 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66cf09-0d7a-467f-9fc0-ca8a62a959d4 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:47.983526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-fe66cf09-0d7a-467f-9fc0-ca8a62a959d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:48.053357 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752b459-b752-4b65-9769-4d950d186da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:48.053674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752b459-b752-4b65-9769-4d950d186da2 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:48.054152 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752b459-b752-4b65-9769-4d950d186da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:48.054513 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752b459-b752-4b65-9769-4d950d186da2 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:48.055581 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f752b459-b752-4b65-9769-4d950d186da2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:48.058583 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f752b459-b752-4b65-9769-4d950d186da2 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:48.060596 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66cf09-0d7a-467f-9fc0-ca8a62a959d4 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:48.065300 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ca6124-6b6c-4cdb-95b8-64b976492f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:48.066096 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca6124-6b6c-4cdb-95b8-64b976492f4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:48.066383 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca6124-6b6c-4cdb-95b8-64b976492f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:48.066659 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca6124-6b6c-4cdb-95b8-64b976492f4d admin admin] PATH_INFO: `/v3/projects/4360b68c2800498e9eb31bc19e54d5e2` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:48.067848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ca6124-6b6c-4cdb-95b8-64b976492f4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4360b68c2800498e9eb31bc19e54d5e2)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:48.070318 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ca6124-6b6c-4cdb-95b8-64b976492f4d admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:48.079989 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:48.088150 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1065/2153] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:47 2022] POST /identity/v3/users => generated 290 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:48.089845 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227c3dd-2cb8-466c-85ec-54011fd21688 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:48.090313 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227c3dd-2cb8-466c-85ec-54011fd21688 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:48.090459 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227c3dd-2cb8-466c-85ec-54011fd21688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:48.114549 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-f227c3dd-2cb8-466c-85ec-54011fd21688 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:48.116017 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f227c3dd-2cb8-466c-85ec-54011fd21688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:48.128598 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-51ca6124-6b6c-4cdb-95b8-64b976492f4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4360b68c2800498e9eb31bc19e54d5e2', 'request_id': 'req-51ca6124-6b6c-4cdb-95b8-64b976492f4d'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:48.136965 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-51ca6124-6b6c-4cdb-95b8-64b976492f4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4360b68c2800498e9eb31bc19e54d5e2', 'request_id': 'req-51ca6124-6b6c-4cdb-95b8-64b976492f4d'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:48.142756 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-51ca6124-6b6c-4cdb-95b8-64b976492f4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4360b68c2800498e9eb31bc19e54d5e2', 'request_id': 'req-51ca6124-6b6c-4cdb-95b8-64b976492f4d'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:48.152455 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:48.157826 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-51ca6124-6b6c-4cdb-95b8-64b976492f4d admin admin] The token cache is being invalidate because project 4360b68c2800498e9eb31bc19e54d5e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:48.158232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-51ca6124-6b6c-4cdb-95b8-64b976492f4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:48.159026 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-51ca6124-6b6c-4cdb-95b8-64b976492f4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:48.161256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1089/2154] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:47 2022] DELETE /identity/v3/projects/4360b68c2800498e9eb31bc19e54d5e2 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:48.162303 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ca6124-6b6c-4cdb-95b8-64b976492f4d admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:48.163672 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-51ca6124-6b6c-4cdb-95b8-64b976492f4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:48.185530 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1066/2155] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:48.191907 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e938ee8-b2b3-4dd2-8bd0-a935fb2afa1f None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:48.191907 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e938ee8-b2b3-4dd2-8bd0-a935fb2afa1f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:48.191907 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e938ee8-b2b3-4dd2-8bd0-a935fb2afa1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:48.207058 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-4e938ee8-b2b3-4dd2-8bd0-a935fb2afa1f None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:48.208297 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-4e938ee8-b2b3-4dd2-8bd0-a935fb2afa1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:48.216786 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1067/2156] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:48.217989 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcb54b-a2aa-4f8f-9413-b3a0bce086d1 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:48.218331 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcb54b-a2aa-4f8f-9413-b3a0bce086d1 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:48.218613 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcb54b-a2aa-4f8f-9413-b3a0bce086d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:48.229696 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ca6124-6b6c-4cdb-95b8-64b976492f4d admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:48.234442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d16ec-11b2-478b-a80c-1edafab3d24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:48.235070 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d16ec-11b2-478b-a80c-1edafab3d24c admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:48.235356 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d16ec-11b2-478b-a80c-1edafab3d24c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:48.235637 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d16ec-11b2-478b-a80c-1edafab3d24c admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:48.236344 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d16ec-11b2-478b-a80c-1edafab3d24c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:48.237955 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d16ec-11b2-478b-a80c-1edafab3d24c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:48.238085 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-efdcb54b-a2aa-4f8f-9413-b3a0bce086d1 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:48.239248 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-efdcb54b-a2aa-4f8f-9413-b3a0bce086d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:48.248123 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1068/2157] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:48.248709 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1090/2158] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:48 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:48.249334 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277a335-6888-4b9d-b851-946b31933f25 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:48.249624 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277a335-6888-4b9d-b851-946b31933f25 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:48.249908 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277a335-6888-4b9d-b851-946b31933f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:48.252357 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d16ec-11b2-478b-a80c-1edafab3d24c admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:48.263483 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d16ec-11b2-478b-a80c-1edafab3d24c admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:48.269396 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ee7d34-5bd2-4f22-b5c5-57b0a9bacf1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:48.270155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee7d34-5bd2-4f22-b5c5-57b0a9bacf1a admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:48.270458 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee7d34-5bd2-4f22-b5c5-57b0a9bacf1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:48.270739 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee7d34-5bd2-4f22-b5c5-57b0a9bacf1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:48.271549 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ee7d34-5bd2-4f22-b5c5-57b0a9bacf1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:48.272047 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-4277a335-6888-4b9d-b851-946b31933f25 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:48.273171 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ee7d34-5bd2-4f22-b5c5-57b0a9bacf1a admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:48.273283 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-4277a335-6888-4b9d-b851-946b31933f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:48.281245 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1069/2159] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:48.282061 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277a335-6888-4b9d-b851-946b31933f25 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:48.286658 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1091/2160] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 04:22:48.286658 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ee7d34-5bd2-4f22-b5c5-57b0a9bacf1a admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:48.292292 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277a335-6888-4b9d-b851-946b31933f25 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:48.295407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ee7d34-5bd2-4f22-b5c5-57b0a9bacf1a admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:48.297258 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7843d06d-0fae-4055-ad81-fe69be91fc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:48.297861 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843d06d-0fae-4055-ad81-fe69be91fc35 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:48.298163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843d06d-0fae-4055-ad81-fe69be91fc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:48.298433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843d06d-0fae-4055-ad81-fe69be91fc35 admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:48.299127 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7843d06d-0fae-4055-ad81-fe69be91fc35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:22:48.299427 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7843d06d-0fae-4055-ad81-fe69be91fc35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:48.300058 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc150545-79e2-4e27-b635-691024819e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:48.300676 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc150545-79e2-4e27-b635-691024819e46 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:48.300959 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc150545-79e2-4e27-b635-691024819e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:48.301063 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7843d06d-0fae-4055-ad81-fe69be91fc35 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:48.301229 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc150545-79e2-4e27-b635-691024819e46 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:48.301921 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc150545-79e2-4e27-b635-691024819e46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:22:48.302262 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc150545-79e2-4e27-b635-691024819e46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:48.303905 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc150545-79e2-4e27-b635-691024819e46 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:48.313129 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1070/2161] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:22:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:22:48.314007 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7843d06d-0fae-4055-ad81-fe69be91fc35 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:48.320399 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1092/2162] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:22:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:22:48.321103 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc150545-79e2-4e27-b635-691024819e46 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:48.326120 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7843d06d-0fae-4055-ad81-fe69be91fc35 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:48.332189 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b340b-893d-44cd-8510-1d3b46658af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:48.332865 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b340b-893d-44cd-8510-1d3b46658af8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:48.333152 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b340b-893d-44cd-8510-1d3b46658af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:48.333378 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc150545-79e2-4e27-b635-691024819e46 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:48.333499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b340b-893d-44cd-8510-1d3b46658af8 admin admin] PATH_INFO: `/v3/projects/08719a0958d4413ebb04848e8a31f98b/users/6ed78cd550204fb18cdcee05bbfd69d9/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:48.338485 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef0837-3554-44b0-9277-b9f8d28a1728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:48.339117 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef0837-3554-44b0-9277-b9f8d28a1728 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:48.339398 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef0837-3554-44b0-9277-b9f8d28a1728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:48.339678 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef0837-3554-44b0-9277-b9f8d28a1728 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:48.340345 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ef0837-3554-44b0-9277-b9f8d28a1728 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:48.342259 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ef0837-3554-44b0-9277-b9f8d28a1728 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:48.350068 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b340b-893d-44cd-8510-1d3b46658af8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08719a0958d4413ebb04848e8a31f98b, user_id=6ed78cd550204fb18cdcee05bbfd69d9, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:48.352710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b340b-893d-44cd-8510-1d3b46658af8 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:48.366405 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1071/2163] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:48 2022] PUT /identity/v3/projects/08719a0958d4413ebb04848e8a31f98b/users/6ed78cd550204fb18cdcee05bbfd69d9/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:48.367326 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b340b-893d-44cd-8510-1d3b46658af8 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:48.380327 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1093/2164] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:48 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:48.381597 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b93eca-94a6-4a54-99a5-d663d42d0f14 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:48.381905 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b93eca-94a6-4a54-99a5-d663d42d0f14 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:48.382199 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b93eca-94a6-4a54-99a5-d663d42d0f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:48.412018 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-54b93eca-94a6-4a54-99a5-d663d42d0f14 None None] MFA Rules not processed for user `6ed78cd550204fb18cdcee05bbfd69d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:48.413251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-54b93eca-94a6-4a54-99a5-d663d42d0f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:48.441867 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b340b-893d-44cd-8510-1d3b46658af8 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:48.480799 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1094/2165] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:48.481702 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b93eca-94a6-4a54-99a5-d663d42d0f14 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:48.489694 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:48.501561 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa884e-5c00-45f4-b79e-78a04a48da66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:48.502230 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa884e-5c00-45f4-b79e-78a04a48da66 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:48.502520 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa884e-5c00-45f4-b79e-78a04a48da66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:48.502800 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa884e-5c00-45f4-b79e-78a04a48da66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:48.503473 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aa884e-5c00-45f4-b79e-78a04a48da66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:48.505381 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aa884e-5c00-45f4-b79e-78a04a48da66 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:48.524400 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:48.536218 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1072/2166] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:48 2022] POST /identity/v3/projects => generated 367 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:48.537248 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa884e-5c00-45f4-b79e-78a04a48da66 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:48.552742 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b93eca-94a6-4a54-99a5-d663d42d0f14 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:48.607237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa884e-5c00-45f4-b79e-78a04a48da66 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:48.612288 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887e5e4-f143-476c-b952-f816d1b015e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:48.613029 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887e5e4-f143-476c-b952-f816d1b015e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:48.613312 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887e5e4-f143-476c-b952-f816d1b015e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:48.613584 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887e5e4-f143-476c-b952-f816d1b015e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:48.614284 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c887e5e4-f143-476c-b952-f816d1b015e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:48.616345 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c887e5e4-f143-476c-b952-f816d1b015e5 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:48.617989 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79128bd0-ef7a-490c-ae64-e96bb7e84c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:48.618628 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79128bd0-ef7a-490c-ae64-e96bb7e84c18 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:48.619479 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79128bd0-ef7a-490c-ae64-e96bb7e84c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:48.619792 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79128bd0-ef7a-490c-ae64-e96bb7e84c18 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:48.621056 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79128bd0-ef7a-490c-ae64-e96bb7e84c18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:48.623737 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79128bd0-ef7a-490c-ae64-e96bb7e84c18 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:48.649948 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1095/2168] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:48 2022] POST /identity/v3/users => generated 286 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:48.649948 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79128bd0-ef7a-490c-ae64-e96bb7e84c18 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:48.650166 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1073/2167] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:48 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:48.650166 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887e5e4-f143-476c-b952-f816d1b015e5 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:48.661388 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79128bd0-ef7a-490c-ae64-e96bb7e84c18 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:48.715937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887e5e4-f143-476c-b952-f816d1b015e5 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:48.720815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79db23-ab35-4464-a1eb-4a9c5acc05ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:48.721442 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79db23-ab35-4464-a1eb-4a9c5acc05ad admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:48.721730 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79db23-ab35-4464-a1eb-4a9c5acc05ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:48.722032 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79db23-ab35-4464-a1eb-4a9c5acc05ad admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:48.722709 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a79db23-ab35-4464-a1eb-4a9c5acc05ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:48.724664 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a79db23-ab35-4464-a1eb-4a9c5acc05ad admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:48.733029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121887bf-340e-4510-bd63-a1b3b13d05ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:48.733659 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121887bf-340e-4510-bd63-a1b3b13d05ca admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:48.733944 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121887bf-340e-4510-bd63-a1b3b13d05ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:48.734252 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121887bf-340e-4510-bd63-a1b3b13d05ca admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:48.734932 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121887bf-340e-4510-bd63-a1b3b13d05ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:48.737262 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121887bf-340e-4510-bd63-a1b3b13d05ca admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:48.748015 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1074/2169] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:48 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:22:48.752221 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79db23-ab35-4464-a1eb-4a9c5acc05ad admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:48.760739 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1096/2170] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:48 2022] POST /identity/v3/users => generated 284 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:48.761474 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121887bf-340e-4510-bd63-a1b3b13d05ca admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:48.762261 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79db23-ab35-4464-a1eb-4a9c5acc05ad admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:48.767282 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e63d02-be98-4215-941f-5d6165d43818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:48.768511 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e63d02-be98-4215-941f-5d6165d43818 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:48.768802 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e63d02-be98-4215-941f-5d6165d43818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:48.769077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e63d02-be98-4215-941f-5d6165d43818 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:48.769804 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e63d02-be98-4215-941f-5d6165d43818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:48.771163 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121887bf-340e-4510-bd63-a1b3b13d05ca admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:48.771491 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e63d02-be98-4215-941f-5d6165d43818 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:48.776903 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf265669-706f-4133-b847-914a8faed384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:48.777513 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf265669-706f-4133-b847-914a8faed384 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:48.778308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf265669-706f-4133-b847-914a8faed384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:48.778308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf265669-706f-4133-b847-914a8faed384 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:48.778905 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf265669-706f-4133-b847-914a8faed384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:48.780490 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf265669-706f-4133-b847-914a8faed384 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:48.782972 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1075/2171] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:48.783810 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e63d02-be98-4215-941f-5d6165d43818 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:48.793851 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e63d02-be98-4215-941f-5d6165d43818 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:48.794304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1097/2172] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:48.795209 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf265669-706f-4133-b847-914a8faed384 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:48.798551 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f9594-3338-4827-90b1-7eeeba7ca47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:48.799162 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f9594-3338-4827-90b1-7eeeba7ca47e admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:48.799445 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f9594-3338-4827-90b1-7eeeba7ca47e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:48.799725 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f9594-3338-4827-90b1-7eeeba7ca47e admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:48.800425 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891f9594-3338-4827-90b1-7eeeba7ca47e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:48.802023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891f9594-3338-4827-90b1-7eeeba7ca47e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:48.807305 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf265669-706f-4133-b847-914a8faed384 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:48.812999 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493bb271-b80f-40e3-9878-d4313d149730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:48.813712 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493bb271-b80f-40e3-9878-d4313d149730 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:48.813941 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1076/2173] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:48.815034 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f9594-3338-4827-90b1-7eeeba7ca47e admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:48.817063 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493bb271-b80f-40e3-9878-d4313d149730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:48.817342 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493bb271-b80f-40e3-9878-d4313d149730 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:48.818383 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493bb271-b80f-40e3-9878-d4313d149730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:48.820223 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493bb271-b80f-40e3-9878-d4313d149730 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:48.827698 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f9594-3338-4827-90b1-7eeeba7ca47e admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:48.834406 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1098/2174] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:48.834406 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493bb271-b80f-40e3-9878-d4313d149730 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:48.835707 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f5e49-cbc9-4ae9-a96a-28f6c1b9595f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:48.836394 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f5e49-cbc9-4ae9-a96a-28f6c1b9595f admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:48.836736 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f5e49-cbc9-4ae9-a96a-28f6c1b9595f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:48.837019 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f5e49-cbc9-4ae9-a96a-28f6c1b9595f admin admin] PATH_INFO: `/v3/projects/7ea69486c9d44c9d8b875ec143fdf345/users/59584830d4de4ecdbce15ff8814a7a6b/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:48.843445 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493bb271-b80f-40e3-9878-d4313d149730 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:48.849139 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686653d0-589e-4dd4-9869-ba7caa2d3f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:48.849770 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686653d0-589e-4dd4-9869-ba7caa2d3f9f admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:48.850067 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686653d0-589e-4dd4-9869-ba7caa2d3f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:48.850345 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686653d0-589e-4dd4-9869-ba7caa2d3f9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:48.851042 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686653d0-589e-4dd4-9869-ba7caa2d3f9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:48.852614 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686653d0-589e-4dd4-9869-ba7caa2d3f9f admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:48.861965 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f5e49-cbc9-4ae9-a96a-28f6c1b9595f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ea69486c9d44c9d8b875ec143fdf345, user_id=59584830d4de4ecdbce15ff8814a7a6b, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:48.864670 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f5e49-cbc9-4ae9-a96a-28f6c1b9595f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:48.865143 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1099/2175] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:48.865832 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686653d0-589e-4dd4-9869-ba7caa2d3f9f admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:48.876599 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686653d0-589e-4dd4-9869-ba7caa2d3f9f admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:48.883119 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1077/2176] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:48 2022] PUT /identity/v3/projects/7ea69486c9d44c9d8b875ec143fdf345/users/59584830d4de4ecdbce15ff8814a7a6b/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:48.886576 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f5e49-cbc9-4ae9-a96a-28f6c1b9595f admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:48.890930 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:48.924011 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:48.979924 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f483c9f-ca43-4c9a-bd6b-9ecacc516539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:48.981161 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f483c9f-ca43-4c9a-bd6b-9ecacc516539 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:48.982139 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f483c9f-ca43-4c9a-bd6b-9ecacc516539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:48.982424 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f483c9f-ca43-4c9a-bd6b-9ecacc516539 admin admin] PATH_INFO: `/v3/projects/18ae062d31a1483ea9b6bdc75ee93b20/users/615ba049b1294439bdd1e01ee90a5dc4/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:48.988301 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f5e49-cbc9-4ae9-a96a-28f6c1b9595f admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:48.994282 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283d813-f8fb-4fe7-b996-dac5de3a30e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:48.995740 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283d813-f8fb-4fe7-b996-dac5de3a30e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:48.996303 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283d813-f8fb-4fe7-b996-dac5de3a30e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:48.996826 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283d813-f8fb-4fe7-b996-dac5de3a30e6 admin admin] PATH_INFO: `/v3/projects/7667031f3bd942c795cf3238ed9fbcf2/users/16a4f447415341e49549de51a12645b0/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:49.003803 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f483c9f-ca43-4c9a-bd6b-9ecacc516539 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18ae062d31a1483ea9b6bdc75ee93b20, user_id=615ba049b1294439bdd1e01ee90a5dc4, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:49.006603 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f483c9f-ca43-4c9a-bd6b-9ecacc516539 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:49.018543 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1283d813-f8fb-4fe7-b996-dac5de3a30e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7667031f3bd942c795cf3238ed9fbcf2, user_id=16a4f447415341e49549de51a12645b0, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:49.021941 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1283d813-f8fb-4fe7-b996-dac5de3a30e6 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:49.025170 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1100/2177] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:48 2022] PUT /identity/v3/projects/18ae062d31a1483ea9b6bdc75ee93b20/users/615ba049b1294439bdd1e01ee90a5dc4/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:49.025784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f483c9f-ca43-4c9a-bd6b-9ecacc516539 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:49.054443 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1078/2178] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:48 2022] PUT /identity/v3/projects/7667031f3bd942c795cf3238ed9fbcf2/users/16a4f447415341e49549de51a12645b0/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:49.056047 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13505c-a868-4cd4-a8cc-9c6bbf3f01cd None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:49.056335 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13505c-a868-4cd4-a8cc-9c6bbf3f01cd None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:49.056626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13505c-a868-4cd4-a8cc-9c6bbf3f01cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:49.102898 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-1a13505c-a868-4cd4-a8cc-9c6bbf3f01cd None None] MFA Rules not processed for user `615ba049b1294439bdd1e01ee90a5dc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:49.104119 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-1a13505c-a868-4cd4-a8cc-9c6bbf3f01cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:49.137895 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f483c9f-ca43-4c9a-bd6b-9ecacc516539 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:49.213306 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1079/2179] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:49.216104 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8cba1f-0672-4c8e-8ffb-47b00de0516d None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:49.216461 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8cba1f-0672-4c8e-8ffb-47b00de0516d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:49.216949 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8cba1f-0672-4c8e-8ffb-47b00de0516d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:49.232872 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2ec2a-5f98-4e81-9d2e-6680eedebf9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:49.233612 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2ec2a-5f98-4e81-9d2e-6680eedebf9b admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:49.234012 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2ec2a-5f98-4e81-9d2e-6680eedebf9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:49.234335 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2ec2a-5f98-4e81-9d2e-6680eedebf9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:49.235124 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f2ec2a-5f98-4e81-9d2e-6680eedebf9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:49.236946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f2ec2a-5f98-4e81-9d2e-6680eedebf9b admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:49.252144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1101/2180] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:49.252144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2ec2a-5f98-4e81-9d2e-6680eedebf9b admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:49.252144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-44f2ec2a-5f98-4e81-9d2e-6680eedebf9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:49.256987 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-bd8cba1f-0672-4c8e-8ffb-47b00de0516d None None] MFA Rules not processed for user `16a4f447415341e49549de51a12645b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:49.258284 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-bd8cba1f-0672-4c8e-8ffb-47b00de0516d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:49.332319 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2ec2a-5f98-4e81-9d2e-6680eedebf9b admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:49.339524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa426701-630e-449d-926b-ff1d03b5465e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e98a41a0f304fc2bb36a18a77a2d729', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:49.340274 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa426701-630e-449d-926b-ff1d03b5465e service glance] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:49.340554 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa426701-630e-449d-926b-ff1d03b5465e service glance] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:49.340971 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa426701-630e-449d-926b-ff1d03b5465e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:49.351923 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa426701-630e-449d-926b-ff1d03b5465e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:49.353802 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa426701-630e-449d-926b-ff1d03b5465e service glance] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:49.359244 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1080/2181] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:49 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:49.360809 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8cba1f-0672-4c8e-8ffb-47b00de0516d None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:49.364983 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:49.373767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1102/2182] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:49.374491 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa426701-630e-449d-926b-ff1d03b5465e service glance] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:49.376982 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-aa426701-630e-449d-926b-ff1d03b5465e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:49.378290 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8cba1f-0672-4c8e-8ffb-47b00de0516d None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:49.387111 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db72a9-d7bd-4562-840c-6af4eb8d0eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:49.388054 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db72a9-d7bd-4562-840c-6af4eb8d0eed admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:49.388482 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db72a9-d7bd-4562-840c-6af4eb8d0eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:49.388937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db72a9-d7bd-4562-840c-6af4eb8d0eed admin admin] PATH_INFO: `/v3/projects/7ea69486c9d44c9d8b875ec143fdf345/users/59584830d4de4ecdbce15ff8814a7a6b/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:49.392206 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db72a9-d7bd-4562-840c-6af4eb8d0eed admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ea69486c9d44c9d8b875ec143fdf345, user_id=59584830d4de4ecdbce15ff8814a7a6b, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:49.397020 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db72a9-d7bd-4562-840c-6af4eb8d0eed admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:49.399809 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:49.404211 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa426701-630e-449d-926b-ff1d03b5465e service glance] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:49.407604 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49821c80-6891-4f79-b996-61cf8155737e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e98a41a0f304fc2bb36a18a77a2d729', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:49.408372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49821c80-6891-4f79-b996-61cf8155737e service glance] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:49.410634 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49821c80-6891-4f79-b996-61cf8155737e service glance] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:49.411385 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49821c80-6891-4f79-b996-61cf8155737e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:49.412702 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1081/2183] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:49 2022] PUT /identity/v3/projects/7ea69486c9d44c9d8b875ec143fdf345/users/59584830d4de4ecdbce15ff8814a7a6b/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:49.427017 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc6064-07c8-45a8-8978-bfa3417aa8c1 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:49.427267 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc6064-07c8-45a8-8978-bfa3417aa8c1 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:49.427623 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc6064-07c8-45a8-8978-bfa3417aa8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:49.429066 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49821c80-6891-4f79-b996-61cf8155737e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:49.431048 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49821c80-6891-4f79-b996-61cf8155737e service glance] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:49.443502 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1103/2184] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:49 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:49.494951 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-67dc6064-07c8-45a8-8978-bfa3417aa8c1 None None] MFA Rules not processed for user `59584830d4de4ecdbce15ff8814a7a6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:49.498323 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-67dc6064-07c8-45a8-8978-bfa3417aa8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:49.616729 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1082/2185] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:49 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:50.044703 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49821c80-6891-4f79-b996-61cf8155737e service glance] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:50.047131 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-49821c80-6891-4f79-b996-61cf8155737e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:50.062049 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc6064-07c8-45a8-8978-bfa3417aa8c1 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:50.062049 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-67dc6064-07c8-45a8-8978-bfa3417aa8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:50.074674 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49821c80-6891-4f79-b996-61cf8155737e service glance] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:50.143852 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc6064-07c8-45a8-8978-bfa3417aa8c1 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:50.148710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a61e6-0245-447e-a244-45600c551585 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:50.149341 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a61e6-0245-447e-a244-45600c551585 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:50.149628 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a61e6-0245-447e-a244-45600c551585 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:50.150598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a61e6-0245-447e-a244-45600c551585 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:50.158502 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8dd43-ac17-40cc-a04f-99d1fd2842d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:50.167241 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a61e6-0245-447e-a244-45600c551585 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:50.169274 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a61e6-0245-447e-a244-45600c551585 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:50.172385 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8dd43-ac17-40cc-a04f-99d1fd2842d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:50.172683 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8dd43-ac17-40cc-a04f-99d1fd2842d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:50.173189 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8dd43-ac17-40cc-a04f-99d1fd2842d7 admin admin] PATH_INFO: `/v3/users/9935fdc20abb4fc09ffb0ea7e055226c` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:50.174733 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf8dd43-ac17-40cc-a04f-99d1fd2842d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9935fdc20abb4fc09ffb0ea7e055226c)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:50.176915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf8dd43-ac17-40cc-a04f-99d1fd2842d7 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:50.184699 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1083/2186] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:50 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 04:22:50.281698 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-bbf8dd43-ac17-40cc-a04f-99d1fd2842d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9935fdc20abb4fc09ffb0ea7e055226c'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:50.282979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-bbf8dd43-ac17-40cc-a04f-99d1fd2842d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9935fdc20abb4fc09ffb0ea7e055226c'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:50.283594 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-bbf8dd43-ac17-40cc-a04f-99d1fd2842d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9935fdc20abb4fc09ffb0ea7e055226c'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:50.297400 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-bbf8dd43-ac17-40cc-a04f-99d1fd2842d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9935fdc20abb4fc09ffb0ea7e055226c'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:50.306126 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-bbf8dd43-ac17-40cc-a04f-99d1fd2842d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9935fdc20abb4fc09ffb0ea7e055226c'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:50.330269 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1104/2187] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:50 2022] DELETE /identity/v3/users/9935fdc20abb4fc09ffb0ea7e055226c => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:50.334453 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563640a-e84c-4171-bf45-ed4f71d840bc None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:50.334649 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563640a-e84c-4171-bf45-ed4f71d840bc None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:50.334947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563640a-e84c-4171-bf45-ed4f71d840bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:50.356488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-7563640a-e84c-4171-bf45-ed4f71d840bc None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:50.358030 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-7563640a-e84c-4171-bf45-ed4f71d840bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:50.378493 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8dd43-ac17-40cc-a04f-99d1fd2842d7 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:50.380859 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-bbf8dd43-ac17-40cc-a04f-99d1fd2842d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:50.436082 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1084/2188] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:50.462024 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563640a-e84c-4171-bf45-ed4f71d840bc None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:50.465312 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-7563640a-e84c-4171-bf45-ed4f71d840bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:50.469212 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8dd43-ac17-40cc-a04f-99d1fd2842d7 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:50.475547 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998bac91-3b5e-4238-8d7e-4fb8220b5803 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:50.476189 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998bac91-3b5e-4238-8d7e-4fb8220b5803 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:50.476645 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998bac91-3b5e-4238-8d7e-4fb8220b5803 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:50.477281 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998bac91-3b5e-4238-8d7e-4fb8220b5803 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:50.481703 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-998bac91-3b5e-4238-8d7e-4fb8220b5803 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:50.509174 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:50.511837 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:50.551192 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563640a-e84c-4171-bf45-ed4f71d840bc None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:50.554879 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83a368-0d7a-462b-ad40-9a25a10d4246 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:50.555493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83a368-0d7a-462b-ad40-9a25a10d4246 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:50.555771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83a368-0d7a-462b-ad40-9a25a10d4246 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:50.556041 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83a368-0d7a-462b-ad40-9a25a10d4246 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:50.563380 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998bac91-3b5e-4238-8d7e-4fb8220b5803 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:50.564832 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998bac91-3b5e-4238-8d7e-4fb8220b5803 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:50.565200 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83a368-0d7a-462b-ad40-9a25a10d4246 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:50.566959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83a368-0d7a-462b-ad40-9a25a10d4246 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:50.574777 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1105/2189] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:50.577442 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1085/2190] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:50.907741 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998bac91-3b5e-4238-8d7e-4fb8220b5803 service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:50.909795 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-998bac91-3b5e-4238-8d7e-4fb8220b5803 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:50.949894 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998bac91-3b5e-4238-8d7e-4fb8220b5803 service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:50.960444 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ed3ab-3163-41b4-925e-1bd2f0c3fc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:50.961470 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ed3ab-3163-41b4-925e-1bd2f0c3fc3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:50.961919 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ed3ab-3163-41b4-925e-1bd2f0c3fc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:50.962417 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ed3ab-3163-41b4-925e-1bd2f0c3fc3c admin admin] PATH_INFO: `/v3/projects/36a8c698b177483ab558e0d4afc8ee5d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:50.965719 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46ed3ab-3163-41b4-925e-1bd2f0c3fc3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=36a8c698b177483ab558e0d4afc8ee5d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:50.968016 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46ed3ab-3163-41b4-925e-1bd2f0c3fc3c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:51.030521 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b46ed3ab-3163-41b4-925e-1bd2f0c3fc3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36a8c698b177483ab558e0d4afc8ee5d', 'request_id': 'req-b46ed3ab-3163-41b4-925e-1bd2f0c3fc3c'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:51.037088 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b46ed3ab-3163-41b4-925e-1bd2f0c3fc3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36a8c698b177483ab558e0d4afc8ee5d', 'request_id': 'req-b46ed3ab-3163-41b4-925e-1bd2f0c3fc3c'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:51.044368 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b46ed3ab-3163-41b4-925e-1bd2f0c3fc3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36a8c698b177483ab558e0d4afc8ee5d', 'request_id': 'req-b46ed3ab-3163-41b4-925e-1bd2f0c3fc3c'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:51.057128 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b46ed3ab-3163-41b4-925e-1bd2f0c3fc3c admin admin] The token cache is being invalidate because project 36a8c698b177483ab558e0d4afc8ee5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:51.057452 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b46ed3ab-3163-41b4-925e-1bd2f0c3fc3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:51.058187 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b46ed3ab-3163-41b4-925e-1bd2f0c3fc3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:51.060291 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1106/2191] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:50 2022] DELETE /identity/v3/projects/36a8c698b177483ab558e0d4afc8ee5d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:51.064377 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83a368-0d7a-462b-ad40-9a25a10d4246 service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:51.076932 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83a368-0d7a-462b-ad40-9a25a10d4246 service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:51.134118 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a15d0-712e-47f0-b618-2c33123ef709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:51.134766 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a15d0-712e-47f0-b618-2c33123ef709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:51.135046 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a15d0-712e-47f0-b618-2c33123ef709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:51.135318 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a15d0-712e-47f0-b618-2c33123ef709 admin admin] PATH_INFO: `/v3/users/b894de5959714790b592c65f0afc8202` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:51.136838 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a15d0-712e-47f0-b618-2c33123ef709 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b894de5959714790b592c65f0afc8202)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:51.138804 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a15d0-712e-47f0-b618-2c33123ef709 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:51.185686 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ce8a15d0-712e-47f0-b618-2c33123ef709 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b894de5959714790b592c65f0afc8202'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:51.186404 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ce8a15d0-712e-47f0-b618-2c33123ef709 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b894de5959714790b592c65f0afc8202'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:51.194101 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ce8a15d0-712e-47f0-b618-2c33123ef709 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b894de5959714790b592c65f0afc8202'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:51.206851 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ce8a15d0-712e-47f0-b618-2c33123ef709 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b894de5959714790b592c65f0afc8202'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:51.214643 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ce8a15d0-712e-47f0-b618-2c33123ef709 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b894de5959714790b592c65f0afc8202'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:51.220669 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1086/2192] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:51 2022] DELETE /identity/v3/users/b894de5959714790b592c65f0afc8202 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:22:51.539748 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ed3ab-3163-41b4-925e-1bd2f0c3fc3c admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:51.541219 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b46ed3ab-3163-41b4-925e-1bd2f0c3fc3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:51.554886 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ed3ab-3163-41b4-925e-1bd2f0c3fc3c admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:51.615051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ebb2b1-fce8-4c99-808c-4de11255fb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:51.615692 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ebb2b1-fce8-4c99-808c-4de11255fb23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:51.615976 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ebb2b1-fce8-4c99-808c-4de11255fb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:51.616255 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ebb2b1-fce8-4c99-808c-4de11255fb23 admin admin] PATH_INFO: `/v3/projects/2bd4c85c2d4b4a9a8f0e89defe7d8a20` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:51.617407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ebb2b1-fce8-4c99-808c-4de11255fb23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bd4c85c2d4b4a9a8f0e89defe7d8a20)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:51.619350 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ebb2b1-fce8-4c99-808c-4de11255fb23 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:51.662759 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:51.665822 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-12ebb2b1-fce8-4c99-808c-4de11255fb23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bd4c85c2d4b4a9a8f0e89defe7d8a20', 'request_id': 'req-12ebb2b1-fce8-4c99-808c-4de11255fb23'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:51.672445 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-12ebb2b1-fce8-4c99-808c-4de11255fb23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bd4c85c2d4b4a9a8f0e89defe7d8a20', 'request_id': 'req-12ebb2b1-fce8-4c99-808c-4de11255fb23'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:51.678523 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-12ebb2b1-fce8-4c99-808c-4de11255fb23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bd4c85c2d4b4a9a8f0e89defe7d8a20', 'request_id': 'req-12ebb2b1-fce8-4c99-808c-4de11255fb23'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:51.690442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-12ebb2b1-fce8-4c99-808c-4de11255fb23 admin admin] The token cache is being invalidate because project 2bd4c85c2d4b4a9a8f0e89defe7d8a20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:51.690729 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-12ebb2b1-fce8-4c99-808c-4de11255fb23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:51.691339 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-12ebb2b1-fce8-4c99-808c-4de11255fb23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:51.692824 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1107/2193] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:51 2022] DELETE /identity/v3/projects/2bd4c85c2d4b4a9a8f0e89defe7d8a20 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:51.697059 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a15d0-712e-47f0-b618-2c33123ef709 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:51.698969 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ce8a15d0-712e-47f0-b618-2c33123ef709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:51.711459 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a15d0-712e-47f0-b618-2c33123ef709 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:51.752341 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:51.769257 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f35c2-24b6-4b94-ad7b-0ad35f575816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:51.769904 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f35c2-24b6-4b94-ad7b-0ad35f575816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:51.770217 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f35c2-24b6-4b94-ad7b-0ad35f575816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:51.770490 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f35c2-24b6-4b94-ad7b-0ad35f575816 admin admin] PATH_INFO: `/v3/users/615c6cdc808b489f80c7dc53c3ace941` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:51.771639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f35c2-24b6-4b94-ad7b-0ad35f575816 admin admin] RBAC: Authorizing `identity:delete_user(user_id=615c6cdc808b489f80c7dc53c3ace941)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:51.773600 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f35c2-24b6-4b94-ad7b-0ad35f575816 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:51.823759 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1d5f35c2-24b6-4b94-ad7b-0ad35f575816 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '615c6cdc808b489f80c7dc53c3ace941'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:51.824467 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1d5f35c2-24b6-4b94-ad7b-0ad35f575816 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '615c6cdc808b489f80c7dc53c3ace941'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:51.831235 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1d5f35c2-24b6-4b94-ad7b-0ad35f575816 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '615c6cdc808b489f80c7dc53c3ace941'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:51.843224 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1d5f35c2-24b6-4b94-ad7b-0ad35f575816 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '615c6cdc808b489f80c7dc53c3ace941'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:51.849826 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1d5f35c2-24b6-4b94-ad7b-0ad35f575816 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '615c6cdc808b489f80c7dc53c3ace941'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:51.855740 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1087/2194] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:51 2022] DELETE /identity/v3/users/615c6cdc808b489f80c7dc53c3ace941 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:52.123947 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ebb2b1-fce8-4c99-808c-4de11255fb23 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:52.125447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-12ebb2b1-fce8-4c99-808c-4de11255fb23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:52.139448 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ebb2b1-fce8-4c99-808c-4de11255fb23 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:52.196377 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e61565-1ecd-4ba7-9a9e-c94119f7c099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:52.197029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e61565-1ecd-4ba7-9a9e-c94119f7c099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:52.197314 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e61565-1ecd-4ba7-9a9e-c94119f7c099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:52.197592 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e61565-1ecd-4ba7-9a9e-c94119f7c099 admin admin] PATH_INFO: `/v3/projects/e7987650d9ee4c74950fd33efd9f2514` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:52.199384 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e61565-1ecd-4ba7-9a9e-c94119f7c099 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7987650d9ee4c74950fd33efd9f2514)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:52.201393 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e61565-1ecd-4ba7-9a9e-c94119f7c099 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:52.247950 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f5e61565-1ecd-4ba7-9a9e-c94119f7c099 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7987650d9ee4c74950fd33efd9f2514', 'request_id': 'req-f5e61565-1ecd-4ba7-9a9e-c94119f7c099'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:52.254263 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f5e61565-1ecd-4ba7-9a9e-c94119f7c099 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7987650d9ee4c74950fd33efd9f2514', 'request_id': 'req-f5e61565-1ecd-4ba7-9a9e-c94119f7c099'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:52.260852 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f5e61565-1ecd-4ba7-9a9e-c94119f7c099 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7987650d9ee4c74950fd33efd9f2514', 'request_id': 'req-f5e61565-1ecd-4ba7-9a9e-c94119f7c099'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:52.275011 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f5e61565-1ecd-4ba7-9a9e-c94119f7c099 admin admin] The token cache is being invalidate because project e7987650d9ee4c74950fd33efd9f2514 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:52.275342 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f5e61565-1ecd-4ba7-9a9e-c94119f7c099 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:52.275968 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f5e61565-1ecd-4ba7-9a9e-c94119f7c099 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:52.278459 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1108/2195] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:52 2022] DELETE /identity/v3/projects/e7987650d9ee4c74950fd33efd9f2514 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:52.284406 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b1c81-3f40-4f2a-b7d7-c6ec9ad9243c None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:52.284891 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b1c81-3f40-4f2a-b7d7-c6ec9ad9243c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:52.285176 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b1c81-3f40-4f2a-b7d7-c6ec9ad9243c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:52.304584 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-6e9b1c81-3f40-4f2a-b7d7-c6ec9ad9243c None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:52.305782 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-6e9b1c81-3f40-4f2a-b7d7-c6ec9ad9243c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:52.365387 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1088/2196] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:22:52.373775 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb51437-b9d3-4573-8b5b-155a6b7ea9c1 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:52.374091 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb51437-b9d3-4573-8b5b-155a6b7ea9c1 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:52.374373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb51437-b9d3-4573-8b5b-155a6b7ea9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:52.395202 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-1cb51437-b9d3-4573-8b5b-155a6b7ea9c1 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:52.396383 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-1cb51437-b9d3-4573-8b5b-155a6b7ea9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:52.405008 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1109/2197] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:52.409053 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b1c81-3f40-4f2a-b7d7-c6ec9ad9243c None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:52.419596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b1c81-3f40-4f2a-b7d7-c6ec9ad9243c None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:52.424294 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52114aa0-5681-46a3-aa83-ad56cac5208c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:52.424911 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52114aa0-5681-46a3-aa83-ad56cac5208c admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:52.425193 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52114aa0-5681-46a3-aa83-ad56cac5208c admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:52.425468 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52114aa0-5681-46a3-aa83-ad56cac5208c admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:52.426211 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52114aa0-5681-46a3-aa83-ad56cac5208c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:22:52.426520 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52114aa0-5681-46a3-aa83-ad56cac5208c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:52.428272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52114aa0-5681-46a3-aa83-ad56cac5208c admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:52.440268 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1089/2198] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:22:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:22:52.444445 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb51437-b9d3-4573-8b5b-155a6b7ea9c1 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:52.457026 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb51437-b9d3-4573-8b5b-155a6b7ea9c1 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:52.462239 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60646a64-4039-4b29-8484-fba82e5d6f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:52.462867 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60646a64-4039-4b29-8484-fba82e5d6f1e admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:52.463146 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60646a64-4039-4b29-8484-fba82e5d6f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:52.463420 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60646a64-4039-4b29-8484-fba82e5d6f1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:52.464789 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60646a64-4039-4b29-8484-fba82e5d6f1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:52.466724 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60646a64-4039-4b29-8484-fba82e5d6f1e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:52.499484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1110/2199] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:52 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:52.500857 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52114aa0-5681-46a3-aa83-ad56cac5208c admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:52.513667 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52114aa0-5681-46a3-aa83-ad56cac5208c admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:52.564799 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:52.577389 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc0fd7-38ba-40e1-b1ed-8a7b349183c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:52.578065 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc0fd7-38ba-40e1-b1ed-8a7b349183c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:52.578371 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc0fd7-38ba-40e1-b1ed-8a7b349183c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:52.578666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc0fd7-38ba-40e1-b1ed-8a7b349183c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:52.579349 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc0fd7-38ba-40e1-b1ed-8a7b349183c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:52.581301 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc0fd7-38ba-40e1-b1ed-8a7b349183c1 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:52.603482 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1090/2200] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:52 2022] POST /identity/v3/users => generated 286 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:22:52.608312 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60646a64-4039-4b29-8484-fba82e5d6f1e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:52.623017 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60646a64-4039-4b29-8484-fba82e5d6f1e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:52.627938 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d5205-3a46-42f2-848b-c3387d747ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:52.628573 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d5205-3a46-42f2-848b-c3387d747ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:52.628854 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d5205-3a46-42f2-848b-c3387d747ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:52.629253 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d5205-3a46-42f2-848b-c3387d747ed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:52.630017 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6d5205-3a46-42f2-848b-c3387d747ed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:52.631654 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6d5205-3a46-42f2-848b-c3387d747ed1 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:52.645662 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1111/2201] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:52.647078 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc0fd7-38ba-40e1-b1ed-8a7b349183c1 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:52.657641 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc0fd7-38ba-40e1-b1ed-8a7b349183c1 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:52.662458 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8513aa-74bb-42b0-bfb5-62e6bfaa85a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:52.663078 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8513aa-74bb-42b0-bfb5-62e6bfaa85a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:52.663363 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8513aa-74bb-42b0-bfb5-62e6bfaa85a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:52.663641 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8513aa-74bb-42b0-bfb5-62e6bfaa85a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:52.664398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8513aa-74bb-42b0-bfb5-62e6bfaa85a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:52.665941 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8513aa-74bb-42b0-bfb5-62e6bfaa85a1 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:52.679930 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1091/2202] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:52.684717 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d5205-3a46-42f2-848b-c3387d747ed1 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:52.695279 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d5205-3a46-42f2-848b-c3387d747ed1 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:52.700922 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087a43e-6a7a-4076-af23-545b7162f296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:52.701584 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087a43e-6a7a-4076-af23-545b7162f296 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:52.701887 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087a43e-6a7a-4076-af23-545b7162f296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:52.702179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087a43e-6a7a-4076-af23-545b7162f296 admin admin] PATH_INFO: `/v3/projects/367dd306bba440ccaff10e20352c5393/users/00cce5451f3c4221813843147399ad5e/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:52.717157 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2087a43e-6a7a-4076-af23-545b7162f296 admin admin] RBAC: Authorizing `identity:create_grant(project_id=367dd306bba440ccaff10e20352c5393, user_id=00cce5451f3c4221813843147399ad5e, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:52.719780 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2087a43e-6a7a-4076-af23-545b7162f296 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:52.728010 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:52.734113 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1112/2203] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:52 2022] PUT /identity/v3/projects/367dd306bba440ccaff10e20352c5393/users/00cce5451f3c4221813843147399ad5e/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:52.739366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b983bad-7934-4be3-b428-a3429ebea806 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:52.739658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b983bad-7934-4be3-b428-a3429ebea806 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:52.739936 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b983bad-7934-4be3-b428-a3429ebea806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:52.758788 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-8b983bad-7934-4be3-b428-a3429ebea806 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:52.759995 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-8b983bad-7934-4be3-b428-a3429ebea806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:52.817578 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1092/2204] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:52.828037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087a43e-6a7a-4076-af23-545b7162f296 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:52.829740 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2087a43e-6a7a-4076-af23-545b7162f296 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:52.886544 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087a43e-6a7a-4076-af23-545b7162f296 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:52.890192 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbeb6a0-5eb8-4d55-87f7-d061d09f4702 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:52.890802 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbeb6a0-5eb8-4d55-87f7-d061d09f4702 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:52.891084 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbeb6a0-5eb8-4d55-87f7-d061d09f4702 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:52.891362 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbeb6a0-5eb8-4d55-87f7-d061d09f4702 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:52.900180 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbeb6a0-5eb8-4d55-87f7-d061d09f4702 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:52.902216 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbeb6a0-5eb8-4d55-87f7-d061d09f4702 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:52.911472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1113/2205] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 04:22:53.768313 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d0e41-8856-43d1-a10b-6c39a4c378f2 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:53.768825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d0e41-8856-43d1-a10b-6c39a4c378f2 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:53.768937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d0e41-8856-43d1-a10b-6c39a4c378f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:53.804050 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-a15d0e41-8856-43d1-a10b-6c39a4c378f2 None None] MFA Rules not processed for user `00cce5451f3c4221813843147399ad5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:53.805274 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-a15d0e41-8856-43d1-a10b-6c39a4c378f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:53.880483 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1093/2206] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:53.884911 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbeb6a0-5eb8-4d55-87f7-d061d09f4702 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:53.894673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbeb6a0-5eb8-4d55-87f7-d061d09f4702 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:53.899174 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af4ffe-b77b-4386-95fa-6db0f2f0f69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:53.899785 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af4ffe-b77b-4386-95fa-6db0f2f0f69e admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:53.900064 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af4ffe-b77b-4386-95fa-6db0f2f0f69e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:53.900335 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af4ffe-b77b-4386-95fa-6db0f2f0f69e admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:53.901035 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04af4ffe-b77b-4386-95fa-6db0f2f0f69e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:53.902990 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04af4ffe-b77b-4386-95fa-6db0f2f0f69e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:53.933474 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1114/2207] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:53 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:53.935028 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d0e41-8856-43d1-a10b-6c39a4c378f2 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:53.946197 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d0e41-8856-43d1-a10b-6c39a4c378f2 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:54.002768 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca8c47-d6be-45b2-b785-8f71f01526b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:54.003406 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca8c47-d6be-45b2-b785-8f71f01526b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:54.003687 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca8c47-d6be-45b2-b785-8f71f01526b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:54.003959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca8c47-d6be-45b2-b785-8f71f01526b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:54.004645 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca8c47-d6be-45b2-b785-8f71f01526b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:54.006625 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca8c47-d6be-45b2-b785-8f71f01526b6 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:54.015344 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:54.030955 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1094/2208] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:53 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:54.034987 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af4ffe-b77b-4386-95fa-6db0f2f0f69e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:54.047105 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af4ffe-b77b-4386-95fa-6db0f2f0f69e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:54.052283 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917753fa-90ab-410a-a621-98912f435cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:54.052934 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917753fa-90ab-410a-a621-98912f435cec admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:54.053217 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917753fa-90ab-410a-a621-98912f435cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:54.053492 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917753fa-90ab-410a-a621-98912f435cec admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:54.054229 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917753fa-90ab-410a-a621-98912f435cec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:54.055935 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917753fa-90ab-410a-a621-98912f435cec admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:54.067956 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1115/2209] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:54.072488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca8c47-d6be-45b2-b785-8f71f01526b6 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:54.083747 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca8c47-d6be-45b2-b785-8f71f01526b6 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:54.088368 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f1d25-89fe-4169-88d3-e002a43c6138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:54.089680 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f1d25-89fe-4169-88d3-e002a43c6138 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:54.089963 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f1d25-89fe-4169-88d3-e002a43c6138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:54.090301 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f1d25-89fe-4169-88d3-e002a43c6138 admin admin] PATH_INFO: `/v3/projects/4c13002ef6634d16ac10f4b5444e1ec3/users/eab1e50b964e497290bdbacc96dd4b4e/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:54.106013 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f1d25-89fe-4169-88d3-e002a43c6138 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c13002ef6634d16ac10f4b5444e1ec3, user_id=eab1e50b964e497290bdbacc96dd4b4e, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:54.108606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f1d25-89fe-4169-88d3-e002a43c6138 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:54.125835 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1095/2210] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:54 2022] PUT /identity/v3/projects/4c13002ef6634d16ac10f4b5444e1ec3/users/eab1e50b964e497290bdbacc96dd4b4e/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:22:54.129841 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917753fa-90ab-410a-a621-98912f435cec admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:54.140761 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917753fa-90ab-410a-a621-98912f435cec admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:54.202662 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e550fec-fb3b-4009-bd81-6e134588a896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:54.203284 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e550fec-fb3b-4009-bd81-6e134588a896 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:54.203569 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e550fec-fb3b-4009-bd81-6e134588a896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:54.203848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e550fec-fb3b-4009-bd81-6e134588a896 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:54.204550 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e550fec-fb3b-4009-bd81-6e134588a896 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:54.206203 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e550fec-fb3b-4009-bd81-6e134588a896 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:54.216084 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1116/2211] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:54.220305 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f1d25-89fe-4169-88d3-e002a43c6138 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:54.230809 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f1d25-89fe-4169-88d3-e002a43c6138 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:54.235487 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ddc0df-11aa-416b-972c-6f6180f83bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:54.236158 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ddc0df-11aa-416b-972c-6f6180f83bcc admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:54.236437 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ddc0df-11aa-416b-972c-6f6180f83bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:54.236709 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ddc0df-11aa-416b-972c-6f6180f83bcc admin admin] PATH_INFO: `/v3/projects/4c13002ef6634d16ac10f4b5444e1ec3/users/eab1e50b964e497290bdbacc96dd4b4e/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:54.238726 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ddc0df-11aa-416b-972c-6f6180f83bcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c13002ef6634d16ac10f4b5444e1ec3, user_id=eab1e50b964e497290bdbacc96dd4b4e, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:54.241349 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ddc0df-11aa-416b-972c-6f6180f83bcc admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:54.252186 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1096/2212] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:54 2022] PUT /identity/v3/projects/4c13002ef6634d16ac10f4b5444e1ec3/users/eab1e50b964e497290bdbacc96dd4b4e/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:55.026980 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf88deb-d4c0-4724-aa3e-a069dba0c268 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:55.027466 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf88deb-d4c0-4724-aa3e-a069dba0c268 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:55.028199 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf88deb-d4c0-4724-aa3e-a069dba0c268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:55.058972 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-9bf88deb-d4c0-4724-aa3e-a069dba0c268 None None] MFA Rules not processed for user `eab1e50b964e497290bdbacc96dd4b4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:55.060308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-9bf88deb-d4c0-4724-aa3e-a069dba0c268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:55.123488 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:55.143909 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1117/2213] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:55 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:55.155229 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ddc0df-11aa-416b-972c-6f6180f83bcc admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:55.156989 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-90ddc0df-11aa-416b-972c-6f6180f83bcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:55.222112 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ddc0df-11aa-416b-972c-6f6180f83bcc admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:55.226729 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b983a2d-60f1-4c8e-ba40-cbf57a338b52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:55.227483 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b983a2d-60f1-4c8e-ba40-cbf57a338b52 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:55.227769 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b983a2d-60f1-4c8e-ba40-cbf57a338b52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:55.228060 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b983a2d-60f1-4c8e-ba40-cbf57a338b52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:55.296425 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b983a2d-60f1-4c8e-ba40-cbf57a338b52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:55.298065 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b983a2d-60f1-4c8e-ba40-cbf57a338b52 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:55.309315 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1097/2214] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:56.857260 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf88deb-d4c0-4724-aa3e-a069dba0c268 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:56.866944 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf88deb-d4c0-4724-aa3e-a069dba0c268 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:56.871612 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f11876-f352-4392-a2cb-813dc742b2b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:56.872222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f11876-f352-4392-a2cb-813dc742b2b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:56.872504 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f11876-f352-4392-a2cb-813dc742b2b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:56.872773 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f11876-f352-4392-a2cb-813dc742b2b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:56.883295 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f11876-f352-4392-a2cb-813dc742b2b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:56.884988 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f11876-f352-4392-a2cb-813dc742b2b8 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:56.896320 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1118/2215] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:56 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:57.000694 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b983a2d-60f1-4c8e-ba40-cbf57a338b52 service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:57.002377 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-4b983a2d-60f1-4c8e-ba40-cbf57a338b52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:57.076230 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b983a2d-60f1-4c8e-ba40-cbf57a338b52 service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:57.082314 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f08924-1531-4aa9-8bcf-bd2795dd793a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:57.083054 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f08924-1531-4aa9-8bcf-bd2795dd793a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:57.083415 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f08924-1531-4aa9-8bcf-bd2795dd793a admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:57.083774 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f08924-1531-4aa9-8bcf-bd2795dd793a admin admin] PATH_INFO: `/v3/users/f694fe71084c4651813a9372bb57f928` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:57.085096 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f08924-1531-4aa9-8bcf-bd2795dd793a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f694fe71084c4651813a9372bb57f928)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:57.087447 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f08924-1531-4aa9-8bcf-bd2795dd793a admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:57.092440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f11876-f352-4392-a2cb-813dc742b2b8 service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:57.094002 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-10f11876-f352-4392-a2cb-813dc742b2b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:57.096704 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:57.111881 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f11876-f352-4392-a2cb-813dc742b2b8 service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:57.119418 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc87980-7a78-4368-bdab-b729eed88dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:57.120623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc87980-7a78-4368-bdab-b729eed88dad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:57.120907 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc87980-7a78-4368-bdab-b729eed88dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:57.121589 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc87980-7a78-4368-bdab-b729eed88dad admin admin] PATH_INFO: `/v3/users/24b5ec62da9645a9bf28a7f6e38b2cf2` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:57.123914 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc87980-7a78-4368-bdab-b729eed88dad admin admin] RBAC: Authorizing `identity:delete_user(user_id=24b5ec62da9645a9bf28a7f6e38b2cf2)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:57.125890 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc87980-7a78-4368-bdab-b729eed88dad admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:57.161565 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f0f08924-1531-4aa9-8bcf-bd2795dd793a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f694fe71084c4651813a9372bb57f928'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:57.162578 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f0f08924-1531-4aa9-8bcf-bd2795dd793a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f694fe71084c4651813a9372bb57f928'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:57.171707 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f0f08924-1531-4aa9-8bcf-bd2795dd793a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f694fe71084c4651813a9372bb57f928'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:57.186492 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f0f08924-1531-4aa9-8bcf-bd2795dd793a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f694fe71084c4651813a9372bb57f928'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:57.193463 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f0f08924-1531-4aa9-8bcf-bd2795dd793a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f694fe71084c4651813a9372bb57f928'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:57.194993 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-abc87980-7a78-4368-bdab-b729eed88dad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24b5ec62da9645a9bf28a7f6e38b2cf2'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:57.195652 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-abc87980-7a78-4368-bdab-b729eed88dad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24b5ec62da9645a9bf28a7f6e38b2cf2'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:57.196254 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-abc87980-7a78-4368-bdab-b729eed88dad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24b5ec62da9645a9bf28a7f6e38b2cf2'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:57.201266 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1098/2216] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:56 2022] DELETE /identity/v3/users/f694fe71084c4651813a9372bb57f928 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:57.205657 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485cfda8-93bb-4c86-b7c5-34db1ea45239 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:57.206049 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485cfda8-93bb-4c86-b7c5-34db1ea45239 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:57.206401 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485cfda8-93bb-4c86-b7c5-34db1ea45239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:57.209438 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-abc87980-7a78-4368-bdab-b729eed88dad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24b5ec62da9645a9bf28a7f6e38b2cf2'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:57.218008 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-abc87980-7a78-4368-bdab-b729eed88dad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24b5ec62da9645a9bf28a7f6e38b2cf2'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:57.228623 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-485cfda8-93bb-4c86-b7c5-34db1ea45239 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:57.230379 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-485cfda8-93bb-4c86-b7c5-34db1ea45239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:57.231865 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1119/2217] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:57 2022] DELETE /identity/v3/users/24b5ec62da9645a9bf28a7f6e38b2cf2 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:22:57.236528 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e4af0-6031-4778-b160-04855c929610 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:57.236821 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e4af0-6031-4778-b160-04855c929610 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:57.237177 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e4af0-6031-4778-b160-04855c929610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:57.257130 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-307e4af0-6031-4778-b160-04855c929610 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:57.258438 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-307e4af0-6031-4778-b160-04855c929610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:57.296199 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1099/2218] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:22:57.306038 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485cfda8-93bb-4c86-b7c5-34db1ea45239 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:57.308165 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-485cfda8-93bb-4c86-b7c5-34db1ea45239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:57.326024 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1120/2219] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:22:57.335619 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e4af0-6031-4778-b160-04855c929610 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:57.338705 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-307e4af0-6031-4778-b160-04855c929610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:57.378049 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485cfda8-93bb-4c86-b7c5-34db1ea45239 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:57.381843 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0983f7c-a0b4-4276-b06c-3fc31b492055 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:57.382768 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0983f7c-a0b4-4276-b06c-3fc31b492055 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:57.382768 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0983f7c-a0b4-4276-b06c-3fc31b492055 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:57.383047 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0983f7c-a0b4-4276-b06c-3fc31b492055 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:57.395857 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0983f7c-a0b4-4276-b06c-3fc31b492055 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:57.397360 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0983f7c-a0b4-4276-b06c-3fc31b492055 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:57.411397 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e4af0-6031-4778-b160-04855c929610 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:57.412762 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1100/2220] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 04:22:57.418626 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e4a50-928c-4e1c-8afe-bcd4dae98efa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:57.419303 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e4a50-928c-4e1c-8afe-bcd4dae98efa service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:57.419590 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e4a50-928c-4e1c-8afe-bcd4dae98efa service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:57.419908 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e4a50-928c-4e1c-8afe-bcd4dae98efa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:57.430480 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:57.437320 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e4a50-928c-4e1c-8afe-bcd4dae98efa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:57.439319 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e4a50-928c-4e1c-8afe-bcd4dae98efa service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:57.472136 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1121/2221] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 04:22:57.885271 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0983f7c-a0b4-4276-b06c-3fc31b492055 service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:57.887671 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d0983f7c-a0b4-4276-b06c-3fc31b492055 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:57.900164 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0983f7c-a0b4-4276-b06c-3fc31b492055 service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:57.905647 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f654a1d7-594c-4228-93df-41fbaf9441a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:57.906391 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654a1d7-594c-4228-93df-41fbaf9441a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:57.906763 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654a1d7-594c-4228-93df-41fbaf9441a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:57.907103 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654a1d7-594c-4228-93df-41fbaf9441a1 admin admin] PATH_INFO: `/v3/users/6aa1364f2f0849acae25076210fb1973` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:57.908336 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f654a1d7-594c-4228-93df-41fbaf9441a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aa1364f2f0849acae25076210fb1973)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:57.910356 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f654a1d7-594c-4228-93df-41fbaf9441a1 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:57.935372 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:57.981685 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f654a1d7-594c-4228-93df-41fbaf9441a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aa1364f2f0849acae25076210fb1973'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:57.982381 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f654a1d7-594c-4228-93df-41fbaf9441a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aa1364f2f0849acae25076210fb1973'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:57.999055 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f654a1d7-594c-4228-93df-41fbaf9441a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aa1364f2f0849acae25076210fb1973'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:58.020887 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f654a1d7-594c-4228-93df-41fbaf9441a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aa1364f2f0849acae25076210fb1973'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:58.032844 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f654a1d7-594c-4228-93df-41fbaf9441a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aa1364f2f0849acae25076210fb1973'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:58.039343 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1101/2222] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:57 2022] DELETE /identity/v3/users/6aa1364f2f0849acae25076210fb1973 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:58.044047 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f654a1d7-594c-4228-93df-41fbaf9441a1 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:58.045446 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30adf3-b81c-41bf-a10f-df5cd929cffd None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:58.046008 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30adf3-b81c-41bf-a10f-df5cd929cffd None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:58.046118 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f654a1d7-594c-4228-93df-41fbaf9441a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:58.046306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30adf3-b81c-41bf-a10f-df5cd929cffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:58.074649 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-2b30adf3-b81c-41bf-a10f-df5cd929cffd None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:58.076993 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2b30adf3-b81c-41bf-a10f-df5cd929cffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:58.146675 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f654a1d7-594c-4228-93df-41fbaf9441a1 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:58.153067 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653bd0b0-0c6f-452e-932e-dfe943175ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:58.153956 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653bd0b0-0c6f-452e-932e-dfe943175ea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:58.154418 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653bd0b0-0c6f-452e-932e-dfe943175ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:58.154850 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653bd0b0-0c6f-452e-932e-dfe943175ea2 admin admin] PATH_INFO: `/v3/projects/7ed1fe0e6b2847e49cbbf7bc1cdfb8f4` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:58.156472 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653bd0b0-0c6f-452e-932e-dfe943175ea2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ed1fe0e6b2847e49cbbf7bc1cdfb8f4)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:58.158629 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1122/2223] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:58.158955 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653bd0b0-0c6f-452e-932e-dfe943175ea2 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:58.159700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b30adf3-b81c-41bf-a10f-df5cd929cffd None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:58.161157 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2b30adf3-b81c-41bf-a10f-df5cd929cffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:58.178583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b30adf3-b81c-41bf-a10f-df5cd929cffd None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:58.190731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e6af7-44bc-40ae-8778-1d020c15152d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:58.192913 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e6af7-44bc-40ae-8778-1d020c15152d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:58.193423 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e6af7-44bc-40ae-8778-1d020c15152d admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:58.193771 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e6af7-44bc-40ae-8778-1d020c15152d admin admin] PATH_INFO: `/v3/projects/6365712030d540bca7469ed29041ee74` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:58.195636 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e6af7-44bc-40ae-8778-1d020c15152d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6365712030d540bca7469ed29041ee74)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:58.197759 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e6af7-44bc-40ae-8778-1d020c15152d admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:58.220811 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-653bd0b0-0c6f-452e-932e-dfe943175ea2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ed1fe0e6b2847e49cbbf7bc1cdfb8f4', 'request_id': 'req-653bd0b0-0c6f-452e-932e-dfe943175ea2'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:58.228615 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-653bd0b0-0c6f-452e-932e-dfe943175ea2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ed1fe0e6b2847e49cbbf7bc1cdfb8f4', 'request_id': 'req-653bd0b0-0c6f-452e-932e-dfe943175ea2'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:58.235865 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-653bd0b0-0c6f-452e-932e-dfe943175ea2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ed1fe0e6b2847e49cbbf7bc1cdfb8f4', 'request_id': 'req-653bd0b0-0c6f-452e-932e-dfe943175ea2'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:58.257185 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-653bd0b0-0c6f-452e-932e-dfe943175ea2 admin admin] The token cache is being invalidate because project 7ed1fe0e6b2847e49cbbf7bc1cdfb8f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:58.257534 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-653bd0b0-0c6f-452e-932e-dfe943175ea2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:58.259010 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-653bd0b0-0c6f-452e-932e-dfe943175ea2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:58.261166 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1102/2224] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:58 2022] DELETE /identity/v3/projects/7ed1fe0e6b2847e49cbbf7bc1cdfb8f4 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:22:58.262188 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653bd0b0-0c6f-452e-932e-dfe943175ea2 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:58.264010 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-653bd0b0-0c6f-452e-932e-dfe943175ea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:58.276562 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f54e6af7-44bc-40ae-8778-1d020c15152d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6365712030d540bca7469ed29041ee74', 'request_id': 'req-f54e6af7-44bc-40ae-8778-1d020c15152d'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:58.283327 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f54e6af7-44bc-40ae-8778-1d020c15152d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6365712030d540bca7469ed29041ee74', 'request_id': 'req-f54e6af7-44bc-40ae-8778-1d020c15152d'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:58.290572 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f54e6af7-44bc-40ae-8778-1d020c15152d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6365712030d540bca7469ed29041ee74', 'request_id': 'req-f54e6af7-44bc-40ae-8778-1d020c15152d'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:58.306719 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f54e6af7-44bc-40ae-8778-1d020c15152d admin admin] The token cache is being invalidate because project 6365712030d540bca7469ed29041ee74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:58.306950 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f54e6af7-44bc-40ae-8778-1d020c15152d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:58.308862 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:58.309757 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f54e6af7-44bc-40ae-8778-1d020c15152d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:58.312797 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1123/2225] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:58 2022] DELETE /identity/v3/projects/6365712030d540bca7469ed29041ee74 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:58.314025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e6af7-44bc-40ae-8778-1d020c15152d admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:58.315662 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f54e6af7-44bc-40ae-8778-1d020c15152d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:58.345077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653bd0b0-0c6f-452e-932e-dfe943175ea2 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:58.409382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e6af7-44bc-40ae-8778-1d020c15152d admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:58.417018 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d101d-f813-4ef1-9761-33215d049395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:58.418022 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d101d-f813-4ef1-9761-33215d049395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:58.418378 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d101d-f813-4ef1-9761-33215d049395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:58.418834 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d101d-f813-4ef1-9761-33215d049395 admin admin] PATH_INFO: `/v3/users/6ed78cd550204fb18cdcee05bbfd69d9` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:58.421870 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d101d-f813-4ef1-9761-33215d049395 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ed78cd550204fb18cdcee05bbfd69d9)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:58.423124 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace27cd-0eca-4e71-b798-cb5d2dcdd55e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:58.423816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace27cd-0eca-4e71-b798-cb5d2dcdd55e service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:58.424098 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace27cd-0eca-4e71-b798-cb5d2dcdd55e service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:58.424370 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace27cd-0eca-4e71-b798-cb5d2dcdd55e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:58.424735 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d101d-f813-4ef1-9761-33215d049395 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:58.426212 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-4ace27cd-0eca-4e71-b798-cb5d2dcdd55e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:58.442250 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace27cd-0eca-4e71-b798-cb5d2dcdd55e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:58.444232 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace27cd-0eca-4e71-b798-cb5d2dcdd55e service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:58.458105 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1103/2226] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:22:58.460231 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc8266-5058-4521-bc65-20ccae45e3ce None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:58.460328 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc8266-5058-4521-bc65-20ccae45e3ce None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:58.460624 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc8266-5058-4521-bc65-20ccae45e3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:58.474688 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:58.490119 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-5edc8266-5058-4521-bc65-20ccae45e3ce None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:58.492352 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-5edc8266-5058-4521-bc65-20ccae45e3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:58.503816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1104/2227] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:58.514395 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae26e4-73c2-4883-b6c8-4d21579d0613 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:58.514861 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae26e4-73c2-4883-b6c8-4d21579d0613 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:58.515324 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae26e4-73c2-4883-b6c8-4d21579d0613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:58.521055 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-701d101d-f813-4ef1-9761-33215d049395 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ed78cd550204fb18cdcee05bbfd69d9'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:58.521871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-701d101d-f813-4ef1-9761-33215d049395 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ed78cd550204fb18cdcee05bbfd69d9'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:58.522793 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-701d101d-f813-4ef1-9761-33215d049395 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ed78cd550204fb18cdcee05bbfd69d9'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:58.537861 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-3bae26e4-73c2-4883-b6c8-4d21579d0613 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:58.538782 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3bae26e4-73c2-4883-b6c8-4d21579d0613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:58.554945 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1105/2228] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:58.562221 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae26e4-73c2-4883-b6c8-4d21579d0613 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:58.566550 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-701d101d-f813-4ef1-9761-33215d049395 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ed78cd550204fb18cdcee05bbfd69d9'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:58.584327 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-701d101d-f813-4ef1-9761-33215d049395 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ed78cd550204fb18cdcee05bbfd69d9'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:58.595685 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae26e4-73c2-4883-b6c8-4d21579d0613 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:58.607380 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a56421-db71-4644-a4db-58ec7056beb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:58.611421 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a56421-db71-4644-a4db-58ec7056beb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:58.613275 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a56421-db71-4644-a4db-58ec7056beb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:58.615215 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a56421-db71-4644-a4db-58ec7056beb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:58.615951 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1124/2229] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:58 2022] DELETE /identity/v3/users/6ed78cd550204fb18cdcee05bbfd69d9 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:58.616340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a56421-db71-4644-a4db-58ec7056beb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:22:58.616767 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a56421-db71-4644-a4db-58ec7056beb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:58.618740 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a56421-db71-4644-a4db-58ec7056beb9 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:58.648997 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1106/2230] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:22:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:22:58.650521 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d101d-f813-4ef1-9761-33215d049395 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:58.757611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d101d-f813-4ef1-9761-33215d049395 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:58.767054 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a5f85-2885-44aa-8dd2-e765a9ad1448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:58.769246 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a5f85-2885-44aa-8dd2-e765a9ad1448 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:58.769531 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a5f85-2885-44aa-8dd2-e765a9ad1448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:58.769846 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a5f85-2885-44aa-8dd2-e765a9ad1448 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:58.770613 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2a5f85-2885-44aa-8dd2-e765a9ad1448 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:58.773164 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2a5f85-2885-44aa-8dd2-e765a9ad1448 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:58.805072 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1125/2231] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:22:58 2022] POST /identity/v3/projects => generated 371 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:58.809679 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a56421-db71-4644-a4db-58ec7056beb9 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:58.829858 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a56421-db71-4644-a4db-58ec7056beb9 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:58.943308 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8321d2-e96f-4aa2-9707-e30a6132ebcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:58.944182 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8321d2-e96f-4aa2-9707-e30a6132ebcd admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:58.944918 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8321d2-e96f-4aa2-9707-e30a6132ebcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:58.945347 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8321d2-e96f-4aa2-9707-e30a6132ebcd admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:58.946220 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8321d2-e96f-4aa2-9707-e30a6132ebcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:58.948767 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8321d2-e96f-4aa2-9707-e30a6132ebcd admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:58.974600 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1107/2232] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:22:58 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:22:58.977955 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a5f85-2885-44aa-8dd2-e765a9ad1448 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:58.990690 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a5f85-2885-44aa-8dd2-e765a9ad1448 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:58.998550 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7686023-10f0-4949-8a6c-bc5119e90e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:59.000131 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7686023-10f0-4949-8a6c-bc5119e90e3c admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:59.000753 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7686023-10f0-4949-8a6c-bc5119e90e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:59.001213 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7686023-10f0-4949-8a6c-bc5119e90e3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:59.002683 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7686023-10f0-4949-8a6c-bc5119e90e3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:59.005761 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7686023-10f0-4949-8a6c-bc5119e90e3c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:59.019148 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1126/2233] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:22:59.023362 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8321d2-e96f-4aa2-9707-e30a6132ebcd admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:59.034491 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8321d2-e96f-4aa2-9707-e30a6132ebcd admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:59.035759 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7686023-10f0-4949-8a6c-bc5119e90e3c admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:59.038936 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b7686023-10f0-4949-8a6c-bc5119e90e3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:59.040125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056b485-d6de-42d5-828e-466462ddb08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:59.040757 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056b485-d6de-42d5-828e-466462ddb08b admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:59.041073 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056b485-d6de-42d5-828e-466462ddb08b admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:59.041309 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056b485-d6de-42d5-828e-466462ddb08b admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:59.042717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056b485-d6de-42d5-828e-466462ddb08b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:59.044874 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056b485-d6de-42d5-828e-466462ddb08b admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:59.071544 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1108/2234] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 04:22:59.081315 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7686023-10f0-4949-8a6c-bc5119e90e3c admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:59.084134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056b485-d6de-42d5-828e-466462ddb08b admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:59.090395 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eba1da-c4e5-41c1-a408-6879eb568570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:59.091672 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eba1da-c4e5-41c1-a408-6879eb568570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:59.092154 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eba1da-c4e5-41c1-a408-6879eb568570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:59.092927 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eba1da-c4e5-41c1-a408-6879eb568570 admin admin] PATH_INFO: `/v3/projects/dddb75e78b394e309f2039e042aaa4a8` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:59.094742 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eba1da-c4e5-41c1-a408-6879eb568570 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dddb75e78b394e309f2039e042aaa4a8)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:59.098302 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eba1da-c4e5-41c1-a408-6879eb568570 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:59.100896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056b485-d6de-42d5-828e-466462ddb08b admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:59.107108 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65540d9-fc5b-4378-8493-a18f72eae370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:59.107792 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65540d9-fc5b-4378-8493-a18f72eae370 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:59.108066 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65540d9-fc5b-4378-8493-a18f72eae370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:59.108589 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65540d9-fc5b-4378-8493-a18f72eae370 admin admin] PATH_INFO: `/v3/projects/923d1dd3cb7a441bb98ce56fc73a9f8e/users/afb594fcd6654ef09b9078d8c5ba21c5/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:59.126291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65540d9-fc5b-4378-8493-a18f72eae370 admin admin] RBAC: Authorizing `identity:create_grant(project_id=923d1dd3cb7a441bb98ce56fc73a9f8e, user_id=afb594fcd6654ef09b9078d8c5ba21c5, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:59.128787 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65540d9-fc5b-4378-8493-a18f72eae370 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:59.165332 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1109/2235] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:22:59 2022] PUT /identity/v3/projects/923d1dd3cb7a441bb98ce56fc73a9f8e/users/afb594fcd6654ef09b9078d8c5ba21c5/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:59.167195 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf8ab2a-72b6-4158-aeea-b7ad4b23a6af None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:59.167479 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf8ab2a-72b6-4158-aeea-b7ad4b23a6af None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:59.167780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf8ab2a-72b6-4158-aeea-b7ad4b23a6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:59.176465 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b3eba1da-c4e5-41c1-a408-6879eb568570 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dddb75e78b394e309f2039e042aaa4a8', 'request_id': 'req-b3eba1da-c4e5-41c1-a408-6879eb568570'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:59.188354 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b3eba1da-c4e5-41c1-a408-6879eb568570 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dddb75e78b394e309f2039e042aaa4a8', 'request_id': 'req-b3eba1da-c4e5-41c1-a408-6879eb568570'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:59.195824 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b3eba1da-c4e5-41c1-a408-6879eb568570 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dddb75e78b394e309f2039e042aaa4a8', 'request_id': 'req-b3eba1da-c4e5-41c1-a408-6879eb568570'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:59.200328 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:59.208655 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-1bf8ab2a-72b6-4158-aeea-b7ad4b23a6af None None] MFA Rules not processed for user `afb594fcd6654ef09b9078d8c5ba21c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:22:59.211089 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-1bf8ab2a-72b6-4158-aeea-b7ad4b23a6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:59.221014 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:22:59.222160 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b3eba1da-c4e5-41c1-a408-6879eb568570 admin admin] The token cache is being invalidate because project dddb75e78b394e309f2039e042aaa4a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:59.222964 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b3eba1da-c4e5-41c1-a408-6879eb568570 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:59.224200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b3eba1da-c4e5-41c1-a408-6879eb568570 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:59.229495 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1127/2236] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:59 2022] DELETE /identity/v3/projects/dddb75e78b394e309f2039e042aaa4a8 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:59.231277 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eba1da-c4e5-41c1-a408-6879eb568570 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:59.234030 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b3eba1da-c4e5-41c1-a408-6879eb568570 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:59.347525 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1110/2237] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:22:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:22:59.357054 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf8ab2a-72b6-4158-aeea-b7ad4b23a6af None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:59.358500 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-1bf8ab2a-72b6-4158-aeea-b7ad4b23a6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:59.370202 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eba1da-c4e5-41c1-a408-6879eb568570 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:59.377798 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f54d1d-64e5-4edd-9e9e-83b4caf853f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:59.379268 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f54d1d-64e5-4edd-9e9e-83b4caf853f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:59.380118 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f54d1d-64e5-4edd-9e9e-83b4caf853f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:59.380118 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f54d1d-64e5-4edd-9e9e-83b4caf853f7 admin admin] PATH_INFO: `/v3/users/e193db74240b4ace8b3b3d17fbf75da4` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:59.381439 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f54d1d-64e5-4edd-9e9e-83b4caf853f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e193db74240b4ace8b3b3d17fbf75da4)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:59.384640 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f54d1d-64e5-4edd-9e9e-83b4caf853f7 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:59.446692 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf8ab2a-72b6-4158-aeea-b7ad4b23a6af None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:59.461569 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ecef0d-afcf-4c67-84b3-4ecd327f217a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:59.461826 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecef0d-afcf-4c67-84b3-4ecd327f217a service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:59.462196 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecef0d-afcf-4c67-84b3-4ecd327f217a service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:59.462401 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecef0d-afcf-4c67-84b3-4ecd327f217a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:59.473313 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ecef0d-afcf-4c67-84b3-4ecd327f217a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:59.474844 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ecef0d-afcf-4c67-84b3-4ecd327f217a service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:59.477204 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-53f54d1d-64e5-4edd-9e9e-83b4caf853f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e193db74240b4ace8b3b3d17fbf75da4'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:59.478016 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-53f54d1d-64e5-4edd-9e9e-83b4caf853f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e193db74240b4ace8b3b3d17fbf75da4'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:59.479669 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-53f54d1d-64e5-4edd-9e9e-83b4caf853f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e193db74240b4ace8b3b3d17fbf75da4'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:59.485325 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1111/2238] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:22:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 04:22:59.486647 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ecef0d-afcf-4c67-84b3-4ecd327f217a service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:59.488574 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-38ecef0d-afcf-4c67-84b3-4ecd327f217a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:59.500987 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-53f54d1d-64e5-4edd-9e9e-83b4caf853f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e193db74240b4ace8b3b3d17fbf75da4'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:59.505517 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ecef0d-afcf-4c67-84b3-4ecd327f217a service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:59.517562 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf38ee-2c2f-4c66-974e-681be55a2ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:22:59.517896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf38ee-2c2f-4c66-974e-681be55a2ba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:22:59.518414 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf38ee-2c2f-4c66-974e-681be55a2ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:22:59.518414 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf38ee-2c2f-4c66-974e-681be55a2ba2 admin admin] PATH_INFO: `/v3/projects/08719a0958d4413ebb04848e8a31f98b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:22:59.519794 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf38ee-2c2f-4c66-974e-681be55a2ba2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08719a0958d4413ebb04848e8a31f98b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:22:59.527276 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf38ee-2c2f-4c66-974e-681be55a2ba2 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:22:59.539505 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-53f54d1d-64e5-4edd-9e9e-83b4caf853f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e193db74240b4ace8b3b3d17fbf75da4'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:59.574712 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1128/2239] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:59 2022] DELETE /identity/v3/users/e193db74240b4ace8b3b3d17fbf75da4 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:59.611933 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4fcf38ee-2c2f-4c66-974e-681be55a2ba2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08719a0958d4413ebb04848e8a31f98b', 'request_id': 'req-4fcf38ee-2c2f-4c66-974e-681be55a2ba2'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:59.622424 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4fcf38ee-2c2f-4c66-974e-681be55a2ba2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08719a0958d4413ebb04848e8a31f98b', 'request_id': 'req-4fcf38ee-2c2f-4c66-974e-681be55a2ba2'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:59.630528 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4fcf38ee-2c2f-4c66-974e-681be55a2ba2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08719a0958d4413ebb04848e8a31f98b', 'request_id': 'req-4fcf38ee-2c2f-4c66-974e-681be55a2ba2'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:59.665469 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4fcf38ee-2c2f-4c66-974e-681be55a2ba2 admin admin] The token cache is being invalidate because project 08719a0958d4413ebb04848e8a31f98b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:22:59.667977 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4fcf38ee-2c2f-4c66-974e-681be55a2ba2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:59.669748 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4fcf38ee-2c2f-4c66-974e-681be55a2ba2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:22:59.674559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1112/2240] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:22:59 2022] DELETE /identity/v3/projects/08719a0958d4413ebb04848e8a31f98b => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:22:59.679083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f54d1d-64e5-4edd-9e9e-83b4caf853f7 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:22:59.680892 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-53f54d1d-64e5-4edd-9e9e-83b4caf853f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:22:59.846818 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f54d1d-64e5-4edd-9e9e-83b4caf853f7 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:22:59.907310 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:00.087818 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603b7e29-df1a-40f8-a8c0-2ff49be125b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:00.089329 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b7e29-df1a-40f8-a8c0-2ff49be125b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:00.090252 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b7e29-df1a-40f8-a8c0-2ff49be125b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:00.091609 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b7e29-df1a-40f8-a8c0-2ff49be125b3 admin admin] PATH_INFO: `/v3/users/59584830d4de4ecdbce15ff8814a7a6b` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:00.094398 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603b7e29-df1a-40f8-a8c0-2ff49be125b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59584830d4de4ecdbce15ff8814a7a6b)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:00.098679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603b7e29-df1a-40f8-a8c0-2ff49be125b3 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:00.269162 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-603b7e29-df1a-40f8-a8c0-2ff49be125b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59584830d4de4ecdbce15ff8814a7a6b'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:00.269997 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-603b7e29-df1a-40f8-a8c0-2ff49be125b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59584830d4de4ecdbce15ff8814a7a6b'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:00.271288 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-603b7e29-df1a-40f8-a8c0-2ff49be125b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59584830d4de4ecdbce15ff8814a7a6b'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:00.305300 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-603b7e29-df1a-40f8-a8c0-2ff49be125b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59584830d4de4ecdbce15ff8814a7a6b'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:00.350069 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-603b7e29-df1a-40f8-a8c0-2ff49be125b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59584830d4de4ecdbce15ff8814a7a6b'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:00.375153 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf38ee-2c2f-4c66-974e-681be55a2ba2 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:00.378500 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-4fcf38ee-2c2f-4c66-974e-681be55a2ba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:00.398335 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1129/2241] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:22:59 2022] DELETE /identity/v3/users/59584830d4de4ecdbce15ff8814a7a6b => generated 0 bytes in 720 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:00.408038 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf38ee-2c2f-4c66-974e-681be55a2ba2 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:00.491927 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:00.524574 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d484448-6c5f-42eb-a04b-a96b07147aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:00.525380 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d484448-6c5f-42eb-a04b-a96b07147aa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:00.525771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d484448-6c5f-42eb-a04b-a96b07147aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:00.526347 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d484448-6c5f-42eb-a04b-a96b07147aa6 admin admin] PATH_INFO: `/v3/projects/91e698a6be8d4ea3a5e87b2b7f76883b` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:00.528299 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d484448-6c5f-42eb-a04b-a96b07147aa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91e698a6be8d4ea3a5e87b2b7f76883b)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:00.530477 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d484448-6c5f-42eb-a04b-a96b07147aa6 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:00.586986 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-5d484448-6c5f-42eb-a04b-a96b07147aa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91e698a6be8d4ea3a5e87b2b7f76883b', 'request_id': 'req-5d484448-6c5f-42eb-a04b-a96b07147aa6'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:00.593410 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-5d484448-6c5f-42eb-a04b-a96b07147aa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91e698a6be8d4ea3a5e87b2b7f76883b', 'request_id': 'req-5d484448-6c5f-42eb-a04b-a96b07147aa6'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:00.599140 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-5d484448-6c5f-42eb-a04b-a96b07147aa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91e698a6be8d4ea3a5e87b2b7f76883b', 'request_id': 'req-5d484448-6c5f-42eb-a04b-a96b07147aa6'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:00.614671 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-5d484448-6c5f-42eb-a04b-a96b07147aa6 admin admin] The token cache is being invalidate because project 91e698a6be8d4ea3a5e87b2b7f76883b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:23:00.614977 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-5d484448-6c5f-42eb-a04b-a96b07147aa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:00.615676 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-5d484448-6c5f-42eb-a04b-a96b07147aa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:00.617203 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1113/2242] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:23:00 2022] DELETE /identity/v3/projects/91e698a6be8d4ea3a5e87b2b7f76883b => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:00.622829 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570510dc-191a-41ce-aef0-31dd04162797 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:00.623176 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570510dc-191a-41ce-aef0-31dd04162797 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:00.623456 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570510dc-191a-41ce-aef0-31dd04162797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:00.643984 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-570510dc-191a-41ce-aef0-31dd04162797 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:00.645197 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-570510dc-191a-41ce-aef0-31dd04162797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:00.714929 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1130/2243] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:00.722636 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b788f-1c7b-45af-a0ce-68471fdcb457 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:00.722931 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b788f-1c7b-45af-a0ce-68471fdcb457 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:00.723204 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b788f-1c7b-45af-a0ce-68471fdcb457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:00.763076 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-276b788f-1c7b-45af-a0ce-68471fdcb457 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:00.765036 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-276b788f-1c7b-45af-a0ce-68471fdcb457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:00.774618 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1114/2244] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:00.779940 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570510dc-191a-41ce-aef0-31dd04162797 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:00.793166 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570510dc-191a-41ce-aef0-31dd04162797 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:00.799108 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9fb5be-1f12-4835-a190-d9164da1d7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:00.800745 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fb5be-1f12-4835-a190-d9164da1d7ea admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:00.801031 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fb5be-1f12-4835-a190-d9164da1d7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:00.801311 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fb5be-1f12-4835-a190-d9164da1d7ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:00.802066 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9fb5be-1f12-4835-a190-d9164da1d7ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:23:00.802705 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9fb5be-1f12-4835-a190-d9164da1d7ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:00.804145 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9fb5be-1f12-4835-a190-d9164da1d7ea admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:00.823094 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b788f-1c7b-45af-a0ce-68471fdcb457 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:00.824784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-276b788f-1c7b-45af-a0ce-68471fdcb457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:00.830440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1131/2245] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:23:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:23:00.832499 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9fb5be-1f12-4835-a190-d9164da1d7ea admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:00.845730 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9fb5be-1f12-4835-a190-d9164da1d7ea admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:00.847744 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b788f-1c7b-45af-a0ce-68471fdcb457 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:00.851166 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3cd90-8079-4203-a819-1d4d1e4ca074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:00.851799 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3cd90-8079-4203-a819-1d4d1e4ca074 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:00.852070 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3cd90-8079-4203-a819-1d4d1e4ca074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:00.852346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3cd90-8079-4203-a819-1d4d1e4ca074 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:00.853236 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3cd90-8079-4203-a819-1d4d1e4ca074 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:00.855991 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3cd90-8079-4203-a819-1d4d1e4ca074 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:00.856529 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80306bb2-2908-4901-b4eb-ec355080ee26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:00.857616 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80306bb2-2908-4901-b4eb-ec355080ee26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:00.858329 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80306bb2-2908-4901-b4eb-ec355080ee26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:00.858707 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80306bb2-2908-4901-b4eb-ec355080ee26 admin admin] PATH_INFO: `/v3/projects/7ea69486c9d44c9d8b875ec143fdf345` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:00.860568 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80306bb2-2908-4901-b4eb-ec355080ee26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ea69486c9d44c9d8b875ec143fdf345)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:00.863355 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80306bb2-2908-4901-b4eb-ec355080ee26 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:00.902015 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1132/2246] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:23:00 2022] POST /identity/v3/projects => generated 367 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:00.904061 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3cd90-8079-4203-a819-1d4d1e4ca074 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:00.913578 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:00.934568 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-80306bb2-2908-4901-b4eb-ec355080ee26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ea69486c9d44c9d8b875ec143fdf345', 'request_id': 'req-80306bb2-2908-4901-b4eb-ec355080ee26'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:00.942326 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-80306bb2-2908-4901-b4eb-ec355080ee26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ea69486c9d44c9d8b875ec143fdf345', 'request_id': 'req-80306bb2-2908-4901-b4eb-ec355080ee26'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:00.958906 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-80306bb2-2908-4901-b4eb-ec355080ee26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ea69486c9d44c9d8b875ec143fdf345', 'request_id': 'req-80306bb2-2908-4901-b4eb-ec355080ee26'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:00.980715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-80306bb2-2908-4901-b4eb-ec355080ee26 admin admin] The token cache is being invalidate because project 7ea69486c9d44c9d8b875ec143fdf345 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:23:00.981122 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-80306bb2-2908-4901-b4eb-ec355080ee26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:00.981872 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-80306bb2-2908-4901-b4eb-ec355080ee26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:00.983639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1115/2247] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:23:00 2022] DELETE /identity/v3/projects/7ea69486c9d44c9d8b875ec143fdf345 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:23:00.989675 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695568d0-b17e-467b-8d4a-438674a12a4e None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:00.989965 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695568d0-b17e-467b-8d4a-438674a12a4e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:00.990276 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695568d0-b17e-467b-8d4a-438674a12a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:01.003302 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3cd90-8079-4203-a819-1d4d1e4ca074 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:01.012463 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-695568d0-b17e-467b-8d4a-438674a12a4e None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:01.013684 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-695568d0-b17e-467b-8d4a-438674a12a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:01.060755 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:01.079633 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31edefa7-455d-49ff-aefd-f0c121af066b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:01.080354 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31edefa7-455d-49ff-aefd-f0c121af066b admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:01.081009 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31edefa7-455d-49ff-aefd-f0c121af066b admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:01.081579 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31edefa7-455d-49ff-aefd-f0c121af066b admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:01.082457 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31edefa7-455d-49ff-aefd-f0c121af066b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:01.085525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31edefa7-455d-49ff-aefd-f0c121af066b admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:01.093060 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1116/2248] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:23:01.101555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6ec0c-abcb-4e1f-8a1a-055506b6c30b None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:01.101791 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6ec0c-abcb-4e1f-8a1a-055506b6c30b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:01.102136 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6ec0c-abcb-4e1f-8a1a-055506b6c30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:01.109449 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1133/2249] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:23:00 2022] POST /identity/v3/users => generated 284 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:01.113445 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31edefa7-455d-49ff-aefd-f0c121af066b admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:01.115193 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-31edefa7-455d-49ff-aefd-f0c121af066b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:01.124779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-c5a6ec0c-abcb-4e1f-8a1a-055506b6c30b None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:01.125966 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c5a6ec0c-abcb-4e1f-8a1a-055506b6c30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:01.127656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31edefa7-455d-49ff-aefd-f0c121af066b admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:01.132322 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175991d-f6e9-4a72-80da-1808f6e2955c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:01.133011 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175991d-f6e9-4a72-80da-1808f6e2955c admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:01.133365 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175991d-f6e9-4a72-80da-1808f6e2955c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:01.133712 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175991d-f6e9-4a72-80da-1808f6e2955c admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:01.134000 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1117/2250] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:23:01.137793 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8175991d-f6e9-4a72-80da-1808f6e2955c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:01.138309 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a6ec0c-abcb-4e1f-8a1a-055506b6c30b None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:01.139441 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8175991d-f6e9-4a72-80da-1808f6e2955c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:01.147223 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a6ec0c-abcb-4e1f-8a1a-055506b6c30b None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:01.152789 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1134/2251] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:01.152925 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c405a8-7da9-4bc4-adc2-3ba06d3cd32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:01.152925 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c405a8-7da9-4bc4-adc2-3ba06d3cd32a admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:01.153191 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c405a8-7da9-4bc4-adc2-3ba06d3cd32a admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:01.153468 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c405a8-7da9-4bc4-adc2-3ba06d3cd32a admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:01.154003 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175991d-f6e9-4a72-80da-1808f6e2955c admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:01.154257 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c405a8-7da9-4bc4-adc2-3ba06d3cd32a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:23:01.154562 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c405a8-7da9-4bc4-adc2-3ba06d3cd32a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:01.156178 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c405a8-7da9-4bc4-adc2-3ba06d3cd32a admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:01.163869 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175991d-f6e9-4a72-80da-1808f6e2955c admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:01.169558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdd8ece-4cd3-402a-a5d1-f735dc53187f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:01.170277 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd8ece-4cd3-402a-a5d1-f735dc53187f admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:01.170574 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd8ece-4cd3-402a-a5d1-f735dc53187f admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:01.170807 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1118/2252] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:23:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:23:01.171451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd8ece-4cd3-402a-a5d1-f735dc53187f admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:01.172578 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd8ece-4cd3-402a-a5d1-f735dc53187f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:01.174242 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd8ece-4cd3-402a-a5d1-f735dc53187f admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:01.175280 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c405a8-7da9-4bc4-adc2-3ba06d3cd32a admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:01.185855 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c405a8-7da9-4bc4-adc2-3ba06d3cd32a admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:01.186040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1135/2253] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:01.190045 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdd8ece-4cd3-402a-a5d1-f735dc53187f admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:01.190774 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510c0bf-3e3e-470a-bcb8-653a39a9f6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:01.191499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510c0bf-3e3e-470a-bcb8-653a39a9f6e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:01.191856 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510c0bf-3e3e-470a-bcb8-653a39a9f6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:01.192204 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510c0bf-3e3e-470a-bcb8-653a39a9f6e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:01.192944 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0510c0bf-3e3e-470a-bcb8-653a39a9f6e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:01.195226 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0510c0bf-3e3e-470a-bcb8-653a39a9f6e4 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:01.200277 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdd8ece-4cd3-402a-a5d1-f735dc53187f admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:01.206706 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5859fcd-5c28-42ed-b0b6-ad8e1dcd9439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:01.207380 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5859fcd-5c28-42ed-b0b6-ad8e1dcd9439 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:01.207666 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5859fcd-5c28-42ed-b0b6-ad8e1dcd9439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:01.207944 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5859fcd-5c28-42ed-b0b6-ad8e1dcd9439 admin admin] PATH_INFO: `/v3/projects/3c8862593dda4d5c9c8ffd075d470d84/users/ce13b58459f741d8a83c91bd1bbef0fd/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:01.228527 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5859fcd-5c28-42ed-b0b6-ad8e1dcd9439 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c8862593dda4d5c9c8ffd075d470d84, user_id=ce13b58459f741d8a83c91bd1bbef0fd, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:01.232979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5859fcd-5c28-42ed-b0b6-ad8e1dcd9439 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:01.237899 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1119/2254] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:23:01 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:01.241672 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510c0bf-3e3e-470a-bcb8-653a39a9f6e4 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:01.259535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1136/2255] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:23:01 2022] PUT /identity/v3/projects/3c8862593dda4d5c9c8ffd075d470d84/users/ce13b58459f741d8a83c91bd1bbef0fd/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:23:01.271655 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd58e4-e24b-4337-9d49-760266788c42 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:01.271975 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd58e4-e24b-4337-9d49-760266788c42 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:01.272223 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd58e4-e24b-4337-9d49-760266788c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:01.307746 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-d8fd58e4-e24b-4337-9d49-760266788c42 None None] MFA Rules not processed for user `ce13b58459f741d8a83c91bd1bbef0fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:01.309583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d8fd58e4-e24b-4337-9d49-760266788c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:01.343616 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510c0bf-3e3e-470a-bcb8-653a39a9f6e4 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:01.372749 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:01.406176 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1137/2256] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:01 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:23:01.410660 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fd58e4-e24b-4337-9d49-760266788c42 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:01.420878 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c217a-8baa-41a5-bd5c-389e9f45a867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:01.421572 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c217a-8baa-41a5-bd5c-389e9f45a867 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:01.421858 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c217a-8baa-41a5-bd5c-389e9f45a867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:01.422166 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c217a-8baa-41a5-bd5c-389e9f45a867 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:01.422844 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762c217a-8baa-41a5-bd5c-389e9f45a867 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:01.424890 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762c217a-8baa-41a5-bd5c-389e9f45a867 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:01.480177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1120/2257] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:23:01 2022] POST /identity/v3/users => generated 292 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:23:01.486643 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c217a-8baa-41a5-bd5c-389e9f45a867 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:01.501906 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c217a-8baa-41a5-bd5c-389e9f45a867 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:01.514686 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3349249-2e00-444f-aa95-37da87cf9f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:01.515353 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3349249-2e00-444f-aa95-37da87cf9f0b admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:01.515633 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3349249-2e00-444f-aa95-37da87cf9f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:01.516098 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3349249-2e00-444f-aa95-37da87cf9f0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:01.516633 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3349249-2e00-444f-aa95-37da87cf9f0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:01.518740 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3349249-2e00-444f-aa95-37da87cf9f0b admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:01.543259 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1121/2258] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:01 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:01.549063 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3349249-2e00-444f-aa95-37da87cf9f0b admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:01.552058 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fd58e4-e24b-4337-9d49-760266788c42 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:01.561156 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3349249-2e00-444f-aa95-37da87cf9f0b admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:01.564843 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d0b016-32c1-4211-a854-46486c746792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:01.566752 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5261c7-ab77-4514-a41f-7445c5c939ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:01.566886 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0b016-32c1-4211-a854-46486c746792 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:01.567389 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5261c7-ab77-4514-a41f-7445c5c939ad admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:01.567671 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5261c7-ab77-4514-a41f-7445c5c939ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:01.567949 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5261c7-ab77-4514-a41f-7445c5c939ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:01.568719 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5261c7-ab77-4514-a41f-7445c5c939ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:01.568937 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0b016-32c1-4211-a854-46486c746792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:01.569903 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0b016-32c1-4211-a854-46486c746792 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:01.570677 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5261c7-ab77-4514-a41f-7445c5c939ad admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:01.571685 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d0b016-32c1-4211-a854-46486c746792 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:01.579644 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d0b016-32c1-4211-a854-46486c746792 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:01.602032 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1122/2259] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:01 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:01.611192 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5261c7-ab77-4514-a41f-7445c5c939ad admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:01.638191 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1138/2260] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:23:01 2022] POST /identity/v3/projects => generated 365 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:23:01.638373 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5261c7-ab77-4514-a41f-7445c5c939ad admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:01.639495 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d0b016-32c1-4211-a854-46486c746792 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:01.652330 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:01.716910 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53215fe8-1897-4d32-a2b4-0579b6fa6995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:01.717894 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53215fe8-1897-4d32-a2b4-0579b6fa6995 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:01.718413 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53215fe8-1897-4d32-a2b4-0579b6fa6995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:01.718890 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53215fe8-1897-4d32-a2b4-0579b6fa6995 admin admin] PATH_INFO: `/v3/projects/4eda5bdd51264b75937c42a129a83020/users/64993a9dda784b699c7366367fde8fb5/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:01.738371 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53215fe8-1897-4d32-a2b4-0579b6fa6995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eda5bdd51264b75937c42a129a83020, user_id=64993a9dda784b699c7366367fde8fb5, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:01.747646 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53215fe8-1897-4d32-a2b4-0579b6fa6995 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:01.781277 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1123/2261] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:23:01 2022] PUT /identity/v3/projects/4eda5bdd51264b75937c42a129a83020/users/64993a9dda784b699c7366367fde8fb5/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:01.798718 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d0b016-32c1-4211-a854-46486c746792 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:01.874656 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20639081-6382-48b8-b985-2d5bd6fe64dc None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:01.874656 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20639081-6382-48b8-b985-2d5bd6fe64dc None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:01.881535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20639081-6382-48b8-b985-2d5bd6fe64dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:01.937795 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-20639081-6382-48b8-b985-2d5bd6fe64dc None None] MFA Rules not processed for user `64993a9dda784b699c7366367fde8fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:01.939048 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-20639081-6382-48b8-b985-2d5bd6fe64dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:01.958224 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11ab49-ac04-4cf8-b693-b27b89e14d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:01.960328 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11ab49-ac04-4cf8-b693-b27b89e14d27 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:01.961509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11ab49-ac04-4cf8-b693-b27b89e14d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:01.961985 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11ab49-ac04-4cf8-b693-b27b89e14d27 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:01.963526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b11ab49-ac04-4cf8-b693-b27b89e14d27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:01.968650 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b11ab49-ac04-4cf8-b693-b27b89e14d27 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:02.014655 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1139/2262] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:23:01 2022] POST /identity/v3/users => generated 289 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:02.018742 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11ab49-ac04-4cf8-b693-b27b89e14d27 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:02.040917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11ab49-ac04-4cf8-b693-b27b89e14d27 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:02.056813 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e5c7e9-e70f-4e7b-ac68-48fcd64fe817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:02.059305 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5c7e9-e70f-4e7b-ac68-48fcd64fe817 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:02.059507 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5c7e9-e70f-4e7b-ac68-48fcd64fe817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:02.059930 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5c7e9-e70f-4e7b-ac68-48fcd64fe817 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:02.060755 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e5c7e9-e70f-4e7b-ac68-48fcd64fe817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:02.065807 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1124/2263] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:02.066473 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20639081-6382-48b8-b985-2d5bd6fe64dc None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:02.070891 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e5c7e9-e70f-4e7b-ac68-48fcd64fe817 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:02.072114 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:02.084304 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20639081-6382-48b8-b985-2d5bd6fe64dc None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:02.091961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ed598-bbc5-474c-98d0-3479d64e5dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:02.094453 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1140/2264] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:02 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:02.097214 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ed598-bbc5-474c-98d0-3479d64e5dee admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:02.097214 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ed598-bbc5-474c-98d0-3479d64e5dee admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:02.097214 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ed598-bbc5-474c-98d0-3479d64e5dee admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:02.099009 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e5c7e9-e70f-4e7b-ac68-48fcd64fe817 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:02.102314 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ed598-bbc5-474c-98d0-3479d64e5dee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:02.106675 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ed598-bbc5-474c-98d0-3479d64e5dee admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:02.114477 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e5c7e9-e70f-4e7b-ac68-48fcd64fe817 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:02.124371 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7018820b-a5da-44ae-ba6e-0678d0661211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:02.125073 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7018820b-a5da-44ae-ba6e-0678d0661211 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:02.125365 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7018820b-a5da-44ae-ba6e-0678d0661211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:02.125648 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7018820b-a5da-44ae-ba6e-0678d0661211 admin admin] PATH_INFO: `/v3/projects/fde1f9ac91c4444294318b5cbc3f441a/users/71323123b47143cda7fcb800d0a686b6/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:02.147255 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7018820b-a5da-44ae-ba6e-0678d0661211 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fde1f9ac91c4444294318b5cbc3f441a, user_id=71323123b47143cda7fcb800d0a686b6, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:02.150209 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7018820b-a5da-44ae-ba6e-0678d0661211 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:02.158908 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1125/2265] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:23:02 2022] POST /identity/v3/projects => generated 381 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:02.163101 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ed598-bbc5-474c-98d0-3479d64e5dee admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:02.171651 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1141/2266] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:23:02 2022] PUT /identity/v3/projects/fde1f9ac91c4444294318b5cbc3f441a/users/71323123b47143cda7fcb800d0a686b6/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:02.175643 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7018820b-a5da-44ae-ba6e-0678d0661211 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:02.290502 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ed598-bbc5-474c-98d0-3479d64e5dee admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:02.293961 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7018820b-a5da-44ae-ba6e-0678d0661211 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:02.298491 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d128ebb0-48e5-4ac7-bf02-7a1647a66217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:02.299317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128ebb0-48e5-4ac7-bf02-7a1647a66217 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:02.299609 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128ebb0-48e5-4ac7-bf02-7a1647a66217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:02.299895 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128ebb0-48e5-4ac7-bf02-7a1647a66217 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:02.300897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d128ebb0-48e5-4ac7-bf02-7a1647a66217 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:02.303125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d128ebb0-48e5-4ac7-bf02-7a1647a66217 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:02.307705 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a409a8-3cdb-4626-bf95-9549911f5383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:02.311546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a409a8-3cdb-4626-bf95-9549911f5383 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:02.311546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a409a8-3cdb-4626-bf95-9549911f5383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:02.312962 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a409a8-3cdb-4626-bf95-9549911f5383 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:02.313957 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a409a8-3cdb-4626-bf95-9549911f5383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:02.315647 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a409a8-3cdb-4626-bf95-9549911f5383 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:02.317620 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:02.338559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1126/2267] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:23:02 2022] POST /identity/v3/users => generated 297 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:23:02.342496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d128ebb0-48e5-4ac7-bf02-7a1647a66217 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:02.343374 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1142/2268] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:02 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:02.347269 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a409a8-3cdb-4626-bf95-9549911f5383 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:02.359784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a409a8-3cdb-4626-bf95-9549911f5383 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:02.370737 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ef286-dc94-4a74-beb6-6311c5e1986a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:02.371352 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d128ebb0-48e5-4ac7-bf02-7a1647a66217 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:02.371884 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ef286-dc94-4a74-beb6-6311c5e1986a admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:02.372222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ef286-dc94-4a74-beb6-6311c5e1986a admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:02.372514 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ef286-dc94-4a74-beb6-6311c5e1986a admin admin] PATH_INFO: `/v3/projects/fde1f9ac91c4444294318b5cbc3f441a/users/71323123b47143cda7fcb800d0a686b6/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:02.389441 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1ef286-dc94-4a74-beb6-6311c5e1986a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fde1f9ac91c4444294318b5cbc3f441a, user_id=71323123b47143cda7fcb800d0a686b6, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:02.389682 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaecbbb-bda4-4431-96d0-a53436410180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:02.389682 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaecbbb-bda4-4431-96d0-a53436410180 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:02.389682 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaecbbb-bda4-4431-96d0-a53436410180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:02.389682 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaecbbb-bda4-4431-96d0-a53436410180 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:02.390127 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaecbbb-bda4-4431-96d0-a53436410180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:02.390210 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1ef286-dc94-4a74-beb6-6311c5e1986a admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:02.390297 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaecbbb-bda4-4431-96d0-a53436410180 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:02.398021 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1127/2269] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:02 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 04:23:02.403481 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaecbbb-bda4-4431-96d0-a53436410180 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:02.409389 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1143/2270] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:23:02 2022] PUT /identity/v3/projects/fde1f9ac91c4444294318b5cbc3f441a/users/71323123b47143cda7fcb800d0a686b6/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:02.412888 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaecbbb-bda4-4431-96d0-a53436410180 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:02.419737 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742db48c-986d-4e3c-817d-9bae0108ee72 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:02.420021 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742db48c-986d-4e3c-817d-9bae0108ee72 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:02.420297 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742db48c-986d-4e3c-817d-9bae0108ee72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:02.460766 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-742db48c-986d-4e3c-817d-9bae0108ee72 None None] MFA Rules not processed for user `71323123b47143cda7fcb800d0a686b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:02.463051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-742db48c-986d-4e3c-817d-9bae0108ee72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:02.489147 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa61688-ad66-46a0-aaef-1ff8597c0216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:02.489790 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa61688-ad66-46a0-aaef-1ff8597c0216 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:02.490113 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa61688-ad66-46a0-aaef-1ff8597c0216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:02.490392 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa61688-ad66-46a0-aaef-1ff8597c0216 admin admin] PATH_INFO: `/v3/projects/374345273f37467f9b057268c2884843/users/e9f5e58e189749aebccbb84d9f3f2fb0/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:02.510054 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa61688-ad66-46a0-aaef-1ff8597c0216 admin admin] RBAC: Authorizing `identity:create_grant(project_id=374345273f37467f9b057268c2884843, user_id=e9f5e58e189749aebccbb84d9f3f2fb0, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:02.519498 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa61688-ad66-46a0-aaef-1ff8597c0216 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:02.535415 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1128/2271] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:23:02 2022] PUT /identity/v3/projects/374345273f37467f9b057268c2884843/users/e9f5e58e189749aebccbb84d9f3f2fb0/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:23:02.538779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa61688-ad66-46a0-aaef-1ff8597c0216 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:02.555722 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1144/2272] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:02 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:23:02.565200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742db48c-986d-4e3c-817d-9bae0108ee72 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:02.567047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-742db48c-986d-4e3c-817d-9bae0108ee72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:02.629687 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa61688-ad66-46a0-aaef-1ff8597c0216 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:02.634799 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53791f8d-b0ae-445d-926b-5508b54eb433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:02.635463 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53791f8d-b0ae-445d-926b-5508b54eb433 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:02.635749 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53791f8d-b0ae-445d-926b-5508b54eb433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:02.636215 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53791f8d-b0ae-445d-926b-5508b54eb433 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:02.636930 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53791f8d-b0ae-445d-926b-5508b54eb433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:02.638613 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53791f8d-b0ae-445d-926b-5508b54eb433 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:02.639269 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:02.653492 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1129/2273] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:02 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:02.654819 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53791f8d-b0ae-445d-926b-5508b54eb433 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:02.657047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742db48c-986d-4e3c-817d-9bae0108ee72 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:02.664835 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53791f8d-b0ae-445d-926b-5508b54eb433 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:02.670989 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23159561-036f-44b2-94f9-fbe8eceb6b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:02.672334 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23159561-036f-44b2-94f9-fbe8eceb6b03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:02.672607 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23159561-036f-44b2-94f9-fbe8eceb6b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:02.672884 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23159561-036f-44b2-94f9-fbe8eceb6b03 admin admin] PATH_INFO: `/v3/projects/374345273f37467f9b057268c2884843/users/e9f5e58e189749aebccbb84d9f3f2fb0/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:02.675594 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23159561-036f-44b2-94f9-fbe8eceb6b03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=374345273f37467f9b057268c2884843, user_id=e9f5e58e189749aebccbb84d9f3f2fb0, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:02.678408 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23159561-036f-44b2-94f9-fbe8eceb6b03 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:02.692879 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1130/2274] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:23:02 2022] PUT /identity/v3/projects/374345273f37467f9b057268c2884843/users/e9f5e58e189749aebccbb84d9f3f2fb0/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:23:02.703332 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6fe87b-136a-4973-a997-5a52a19dea3f None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:02.704083 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6fe87b-136a-4973-a997-5a52a19dea3f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:02.704382 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6fe87b-136a-4973-a997-5a52a19dea3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:02.715885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5491443-7bb1-4d4a-b049-47e7c009fc28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:02.716083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5491443-7bb1-4d4a-b049-47e7c009fc28 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:02.716368 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5491443-7bb1-4d4a-b049-47e7c009fc28 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:02.716647 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5491443-7bb1-4d4a-b049-47e7c009fc28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:02.737126 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-5a6fe87b-136a-4973-a997-5a52a19dea3f None None] MFA Rules not processed for user `e9f5e58e189749aebccbb84d9f3f2fb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:02.738492 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-5a6fe87b-136a-4973-a997-5a52a19dea3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:02.780727 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5491443-7bb1-4d4a-b049-47e7c009fc28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:02.781563 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:02.782295 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5491443-7bb1-4d4a-b049-47e7c009fc28 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:02.794678 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1145/2275] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:02 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 04:23:02.816262 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1131/2276] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:23:02.829331 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5491443-7bb1-4d4a-b049-47e7c009fc28 service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:02.830808 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a5491443-7bb1-4d4a-b049-47e7c009fc28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:02.844500 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5491443-7bb1-4d4a-b049-47e7c009fc28 service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:02.882282 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6fe87b-136a-4973-a997-5a52a19dea3f None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:02.902388 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20add61b-9940-4c37-b44e-2b50976ff319 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:02.903022 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20add61b-9940-4c37-b44e-2b50976ff319 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:02.904953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20add61b-9940-4c37-b44e-2b50976ff319 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:02.904953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20add61b-9940-4c37-b44e-2b50976ff319 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:02.920176 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20add61b-9940-4c37-b44e-2b50976ff319 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:02.921684 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20add61b-9940-4c37-b44e-2b50976ff319 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:02.934462 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1146/2277] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:02 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:23:02.956814 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6fe87b-136a-4973-a997-5a52a19dea3f None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:02.961451 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cfca3b-6857-40d5-8dc5-e0153d486539 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:02.962185 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cfca3b-6857-40d5-8dc5-e0153d486539 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:02.962562 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cfca3b-6857-40d5-8dc5-e0153d486539 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:02.962907 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cfca3b-6857-40d5-8dc5-e0153d486539 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:03.040979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20add61b-9940-4c37-b44e-2b50976ff319 service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:03.047637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cfca3b-6857-40d5-8dc5-e0153d486539 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:03.049518 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cfca3b-6857-40d5-8dc5-e0153d486539 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:03.052311 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20add61b-9940-4c37-b44e-2b50976ff319 service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:03.060452 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1132/2278] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:02 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 04:23:03.061055 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d938516-e8a5-4260-8679-5c5c5387c7d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:03.061923 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d938516-e8a5-4260-8679-5c5c5387c7d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:03.062346 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d938516-e8a5-4260-8679-5c5c5387c7d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:03.062689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d938516-e8a5-4260-8679-5c5c5387c7d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:03.120057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:03.134248 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d938516-e8a5-4260-8679-5c5c5387c7d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:03.136216 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d938516-e8a5-4260-8679-5c5c5387c7d3 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:03.149529 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1147/2279] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:23:03.628624 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cfca3b-6857-40d5-8dc5-e0153d486539 service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:03.630146 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-64cfca3b-6857-40d5-8dc5-e0153d486539 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:03.693778 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d938516-e8a5-4260-8679-5c5c5387c7d3 service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:03.717333 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cfca3b-6857-40d5-8dc5-e0153d486539 service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:03.722891 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50efb44-60c7-49d6-b91d-4b817e1ce19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:03.723670 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50efb44-60c7-49d6-b91d-4b817e1ce19b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:03.724059 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50efb44-60c7-49d6-b91d-4b817e1ce19b admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:03.724823 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50efb44-60c7-49d6-b91d-4b817e1ce19b admin admin] PATH_INFO: `/v3/users/afb594fcd6654ef09b9078d8c5ba21c5` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:03.726440 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50efb44-60c7-49d6-b91d-4b817e1ce19b admin admin] RBAC: Authorizing `identity:delete_user(user_id=afb594fcd6654ef09b9078d8c5ba21c5)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:03.729243 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50efb44-60c7-49d6-b91d-4b817e1ce19b admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:03.773101 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:03.795163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-e50efb44-60c7-49d6-b91d-4b817e1ce19b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afb594fcd6654ef09b9078d8c5ba21c5'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:03.796356 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-e50efb44-60c7-49d6-b91d-4b817e1ce19b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afb594fcd6654ef09b9078d8c5ba21c5'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:03.798651 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d938516-e8a5-4260-8679-5c5c5387c7d3 service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:03.804117 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66849d1-07ab-4201-9f87-318851e200df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:03.804749 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66849d1-07ab-4201-9f87-318851e200df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:03.805036 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66849d1-07ab-4201-9f87-318851e200df admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:03.805317 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66849d1-07ab-4201-9f87-318851e200df admin admin] PATH_INFO: `/v3/users/64993a9dda784b699c7366367fde8fb5` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:03.807876 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-e50efb44-60c7-49d6-b91d-4b817e1ce19b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afb594fcd6654ef09b9078d8c5ba21c5'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:03.809155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66849d1-07ab-4201-9f87-318851e200df admin admin] RBAC: Authorizing `identity:delete_user(user_id=64993a9dda784b699c7366367fde8fb5)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:03.813501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66849d1-07ab-4201-9f87-318851e200df admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:03.825448 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-e50efb44-60c7-49d6-b91d-4b817e1ce19b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afb594fcd6654ef09b9078d8c5ba21c5'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:03.842776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-e50efb44-60c7-49d6-b91d-4b817e1ce19b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afb594fcd6654ef09b9078d8c5ba21c5'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:03.851525 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1133/2280] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:23:03 2022] DELETE /identity/v3/users/afb594fcd6654ef09b9078d8c5ba21c5 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:03.858796 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdeef22-543e-44c0-9105-126c543826b4 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:03.860698 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdeef22-543e-44c0-9105-126c543826b4 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:03.860698 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdeef22-543e-44c0-9105-126c543826b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:03.888196 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-0fdeef22-543e-44c0-9105-126c543826b4 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:03.889794 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0fdeef22-543e-44c0-9105-126c543826b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:03.921151 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a66849d1-07ab-4201-9f87-318851e200df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64993a9dda784b699c7366367fde8fb5'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:03.922421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a66849d1-07ab-4201-9f87-318851e200df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64993a9dda784b699c7366367fde8fb5'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:03.924837 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a66849d1-07ab-4201-9f87-318851e200df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64993a9dda784b699c7366367fde8fb5'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:03.946365 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a66849d1-07ab-4201-9f87-318851e200df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64993a9dda784b699c7366367fde8fb5'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:03.957676 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a66849d1-07ab-4201-9f87-318851e200df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64993a9dda784b699c7366367fde8fb5'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:03.974562 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1134/2281] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:03.978950 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1148/2282] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:23:03 2022] DELETE /identity/v3/users/64993a9dda784b699c7366367fde8fb5 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:23:03.981842 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b0195-8b86-496c-a733-d5b10db80b48 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:03.982167 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b0195-8b86-496c-a733-d5b10db80b48 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:03.982447 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b0195-8b86-496c-a733-d5b10db80b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:03.984843 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66849d1-07ab-4201-9f87-318851e200df admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:03.986423 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a66849d1-07ab-4201-9f87-318851e200df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:04.004828 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-047b0195-8b86-496c-a733-d5b10db80b48 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:04.006099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-047b0195-8b86-496c-a733-d5b10db80b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:04.053341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66849d1-07ab-4201-9f87-318851e200df admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:04.057730 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303dd91d-e49c-4653-9564-6bea4d9cb022 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:04.058942 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303dd91d-e49c-4653-9564-6bea4d9cb022 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:04.059553 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303dd91d-e49c-4653-9564-6bea4d9cb022 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:04.060130 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303dd91d-e49c-4653-9564-6bea4d9cb022 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:04.083808 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1135/2283] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:23:04.096213 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:04.096894 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b0195-8b86-496c-a733-d5b10db80b48 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:04.112179 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b0195-8b86-496c-a733-d5b10db80b48 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:04.116347 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd2f90-670f-469d-996e-94594501f9f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:04.117036 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd2f90-670f-469d-996e-94594501f9f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:04.118073 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd2f90-670f-469d-996e-94594501f9f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:04.118235 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd2f90-670f-469d-996e-94594501f9f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:04.130655 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cd2f90-670f-469d-996e-94594501f9f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:04.132439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cd2f90-670f-469d-996e-94594501f9f6 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:04.144690 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1136/2284] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:23:04.157305 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303dd91d-e49c-4653-9564-6bea4d9cb022 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:04.159353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303dd91d-e49c-4653-9564-6bea4d9cb022 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:04.173166 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1149/2285] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 04:23:04.723304 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd2f90-670f-469d-996e-94594501f9f6 service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:04.726409 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b4cd2f90-670f-469d-996e-94594501f9f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:04.751539 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd2f90-670f-469d-996e-94594501f9f6 service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:04.757745 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556bca26-917c-4dad-b6b7-a3df3b627b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:04.758447 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bca26-917c-4dad-b6b7-a3df3b627b78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:04.758779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bca26-917c-4dad-b6b7-a3df3b627b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:04.759063 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bca26-917c-4dad-b6b7-a3df3b627b78 admin admin] PATH_INFO: `/v3/projects/923d1dd3cb7a441bb98ce56fc73a9f8e` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:04.761079 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556bca26-917c-4dad-b6b7-a3df3b627b78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=923d1dd3cb7a441bb98ce56fc73a9f8e)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:04.774742 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556bca26-917c-4dad-b6b7-a3df3b627b78 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:04.826471 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:04.860423 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-556bca26-917c-4dad-b6b7-a3df3b627b78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '923d1dd3cb7a441bb98ce56fc73a9f8e', 'request_id': 'req-556bca26-917c-4dad-b6b7-a3df3b627b78'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:04.868218 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-556bca26-917c-4dad-b6b7-a3df3b627b78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '923d1dd3cb7a441bb98ce56fc73a9f8e', 'request_id': 'req-556bca26-917c-4dad-b6b7-a3df3b627b78'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:04.876969 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-556bca26-917c-4dad-b6b7-a3df3b627b78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '923d1dd3cb7a441bb98ce56fc73a9f8e', 'request_id': 'req-556bca26-917c-4dad-b6b7-a3df3b627b78'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:04.896548 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-556bca26-917c-4dad-b6b7-a3df3b627b78 admin admin] The token cache is being invalidate because project 923d1dd3cb7a441bb98ce56fc73a9f8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:23:04.896885 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-556bca26-917c-4dad-b6b7-a3df3b627b78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:04.898452 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-556bca26-917c-4dad-b6b7-a3df3b627b78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:04.902055 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1137/2286] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:23:04 2022] DELETE /identity/v3/projects/923d1dd3cb7a441bb98ce56fc73a9f8e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:04.907524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932add1d-dece-4af2-815b-fb4c1c7704e1 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:04.908149 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932add1d-dece-4af2-815b-fb4c1c7704e1 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:04.908523 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932add1d-dece-4af2-815b-fb4c1c7704e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:04.937592 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-932add1d-dece-4af2-815b-fb4c1c7704e1 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:04.945677 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-932add1d-dece-4af2-815b-fb4c1c7704e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:05.016082 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556bca26-917c-4dad-b6b7-a3df3b627b78 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:05.026303 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-556bca26-917c-4dad-b6b7-a3df3b627b78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:05.109543 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1150/2287] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:05.111511 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b9816-64c1-43c5-9222-5a9a86eb0b9f None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:05.111666 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b9816-64c1-43c5-9222-5a9a86eb0b9f None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:05.111949 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b9816-64c1-43c5-9222-5a9a86eb0b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:05.137055 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-bc7b9816-64c1-43c5-9222-5a9a86eb0b9f None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:05.138732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-bc7b9816-64c1-43c5-9222-5a9a86eb0b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:05.147463 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1151/2288] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:05.152877 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b9816-64c1-43c5-9222-5a9a86eb0b9f None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:05.162906 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b9816-64c1-43c5-9222-5a9a86eb0b9f None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:05.169844 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf83543b-b60e-45da-8607-a515d2f1832e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:05.170756 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83543b-b60e-45da-8607-a515d2f1832e admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:05.172288 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83543b-b60e-45da-8607-a515d2f1832e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:05.172633 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83543b-b60e-45da-8607-a515d2f1832e admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:05.174406 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf83543b-b60e-45da-8607-a515d2f1832e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:23:05.174818 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf83543b-b60e-45da-8607-a515d2f1832e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:05.176787 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf83543b-b60e-45da-8607-a515d2f1832e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:05.189628 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556bca26-917c-4dad-b6b7-a3df3b627b78 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:05.194631 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1152/2289] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:23:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:23:05.195473 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dabe905-801a-4cc2-a834-94d307941274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:05.196342 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabe905-801a-4cc2-a834-94d307941274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:05.196606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabe905-801a-4cc2-a834-94d307941274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:05.196834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabe905-801a-4cc2-a834-94d307941274 admin admin] PATH_INFO: `/v3/projects/4eda5bdd51264b75937c42a129a83020` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:05.198971 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dabe905-801a-4cc2-a834-94d307941274 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4eda5bdd51264b75937c42a129a83020)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:05.200655 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf83543b-b60e-45da-8607-a515d2f1832e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:05.201040 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dabe905-801a-4cc2-a834-94d307941274 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:05.214322 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf83543b-b60e-45da-8607-a515d2f1832e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:05.220350 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5413f8-8696-45b2-9aca-97a4ce7f2c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:05.221203 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5413f8-8696-45b2-9aca-97a4ce7f2c60 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:05.221668 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5413f8-8696-45b2-9aca-97a4ce7f2c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:05.222142 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5413f8-8696-45b2-9aca-97a4ce7f2c60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:05.223051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5413f8-8696-45b2-9aca-97a4ce7f2c60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:05.227063 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5413f8-8696-45b2-9aca-97a4ce7f2c60 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:05.269626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-8dabe905-801a-4cc2-a834-94d307941274 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4eda5bdd51264b75937c42a129a83020', 'request_id': 'req-8dabe905-801a-4cc2-a834-94d307941274'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:05.271869 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1153/2290] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:23:05 2022] POST /identity/v3/projects => generated 363 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:05.279029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5413f8-8696-45b2-9aca-97a4ce7f2c60 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:05.281754 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-8dabe905-801a-4cc2-a834-94d307941274 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4eda5bdd51264b75937c42a129a83020', 'request_id': 'req-8dabe905-801a-4cc2-a834-94d307941274'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:05.290039 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-8dabe905-801a-4cc2-a834-94d307941274 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4eda5bdd51264b75937c42a129a83020', 'request_id': 'req-8dabe905-801a-4cc2-a834-94d307941274'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:05.309383 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-8dabe905-801a-4cc2-a834-94d307941274 admin admin] The token cache is being invalidate because project 4eda5bdd51264b75937c42a129a83020 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:23:05.309673 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-8dabe905-801a-4cc2-a834-94d307941274 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:05.310371 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-8dabe905-801a-4cc2-a834-94d307941274 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:05.312175 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1138/2291] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:23:05 2022] DELETE /identity/v3/projects/4eda5bdd51264b75937c42a129a83020 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:05.316168 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dabe905-801a-4cc2-a834-94d307941274 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:05.318418 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-8dabe905-801a-4cc2-a834-94d307941274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:05.376134 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5413f8-8696-45b2-9aca-97a4ce7f2c60 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:05.388008 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ded862-8e7b-46b4-a48e-7045cdd67d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:05.389367 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ded862-8e7b-46b4-a48e-7045cdd67d12 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:05.389864 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ded862-8e7b-46b4-a48e-7045cdd67d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:05.390372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ded862-8e7b-46b4-a48e-7045cdd67d12 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:05.391498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ded862-8e7b-46b4-a48e-7045cdd67d12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:05.393646 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ded862-8e7b-46b4-a48e-7045cdd67d12 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:05.412534 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:05.433323 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1154/2292] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:23:05 2022] POST /identity/v3/users => generated 282 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:05.435481 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dabe905-801a-4cc2-a834-94d307941274 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:05.437571 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ded862-8e7b-46b4-a48e-7045cdd67d12 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:05.439797 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a5ded862-8e7b-46b4-a48e-7045cdd67d12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:05.441622 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3a3b69-0e09-48e9-8c98-f728420dc1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:05.442353 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3a3b69-0e09-48e9-8c98-f728420dc1d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:05.442672 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3a3b69-0e09-48e9-8c98-f728420dc1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:05.443011 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3a3b69-0e09-48e9-8c98-f728420dc1d7 admin admin] PATH_INFO: `/v3/users/e9f5e58e189749aebccbb84d9f3f2fb0` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:05.444401 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3a3b69-0e09-48e9-8c98-f728420dc1d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9f5e58e189749aebccbb84d9f3f2fb0)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:05.446879 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3a3b69-0e09-48e9-8c98-f728420dc1d7 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:05.457158 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:05.463443 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ded862-8e7b-46b4-a48e-7045cdd67d12 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:05.472178 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4adb3-e03d-4e1f-905f-eb3e1a51eada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:05.473577 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4adb3-e03d-4e1f-905f-eb3e1a51eada admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:05.473860 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4adb3-e03d-4e1f-905f-eb3e1a51eada admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:05.474270 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4adb3-e03d-4e1f-905f-eb3e1a51eada admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:05.475870 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a4adb3-e03d-4e1f-905f-eb3e1a51eada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:05.477504 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a4adb3-e03d-4e1f-905f-eb3e1a51eada admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:05.494024 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1155/2293] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:05 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 04:23:05.501091 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4adb3-e03d-4e1f-905f-eb3e1a51eada admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:05.523976 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ed3a3b69-0e09-48e9-8c98-f728420dc1d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9f5e58e189749aebccbb84d9f3f2fb0'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:05.524718 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ed3a3b69-0e09-48e9-8c98-f728420dc1d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9f5e58e189749aebccbb84d9f3f2fb0'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:05.526995 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4adb3-e03d-4e1f-905f-eb3e1a51eada admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:05.534327 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23cc30-a9cc-49e3-a356-d881f3e84f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:05.534551 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ed3a3b69-0e09-48e9-8c98-f728420dc1d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9f5e58e189749aebccbb84d9f3f2fb0'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:05.535093 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23cc30-a9cc-49e3-a356-d881f3e84f27 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:05.535417 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23cc30-a9cc-49e3-a356-d881f3e84f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:05.535673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23cc30-a9cc-49e3-a356-d881f3e84f27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:05.536546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23cc30-a9cc-49e3-a356-d881f3e84f27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:05.538915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23cc30-a9cc-49e3-a356-d881f3e84f27 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:05.549721 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ed3a3b69-0e09-48e9-8c98-f728420dc1d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9f5e58e189749aebccbb84d9f3f2fb0'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:05.557650 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ed3a3b69-0e09-48e9-8c98-f728420dc1d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9f5e58e189749aebccbb84d9f3f2fb0'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:05.558559 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1156/2294] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:05 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:05.562955 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23cc30-a9cc-49e3-a356-d881f3e84f27 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:05.564937 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-ce23cc30-a9cc-49e3-a356-d881f3e84f27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:05.567384 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1139/2295] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:23:05 2022] DELETE /identity/v3/users/e9f5e58e189749aebccbb84d9f3f2fb0 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:05.587013 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23cc30-a9cc-49e3-a356-d881f3e84f27 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:05.662643 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1e7db-9987-49e8-a21b-10eae8e64277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:05.663435 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1e7db-9987-49e8-a21b-10eae8e64277 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:05.663797 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1e7db-9987-49e8-a21b-10eae8e64277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:05.664147 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1e7db-9987-49e8-a21b-10eae8e64277 admin admin] PATH_INFO: `/v3/projects/275b579029e14dc3827ab8ccfc992890/users/ded747a48fec476bb98a8cf2a9c871d7/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:05.679970 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb1e7db-9987-49e8-a21b-10eae8e64277 admin admin] RBAC: Authorizing `identity:create_grant(project_id=275b579029e14dc3827ab8ccfc992890, user_id=ded747a48fec476bb98a8cf2a9c871d7, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:05.682982 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb1e7db-9987-49e8-a21b-10eae8e64277 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:05.699103 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1157/2296] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:23:05 2022] PUT /identity/v3/projects/275b579029e14dc3827ab8ccfc992890/users/ded747a48fec476bb98a8cf2a9c871d7/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:05.709469 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc58696-81f6-4efa-ba52-5b51eca71868 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:05.709864 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc58696-81f6-4efa-ba52-5b51eca71868 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:05.710242 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc58696-81f6-4efa-ba52-5b51eca71868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:05.740968 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-abc58696-81f6-4efa-ba52-5b51eca71868 None None] MFA Rules not processed for user `ded747a48fec476bb98a8cf2a9c871d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:05.742228 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-abc58696-81f6-4efa-ba52-5b51eca71868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:05.814248 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1140/2297] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:05 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:05.830816 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1e7db-9987-49e8-a21b-10eae8e64277 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:05.835846 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-4cb1e7db-9987-49e8-a21b-10eae8e64277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:05.936185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1e7db-9987-49e8-a21b-10eae8e64277 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:05.940305 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26478b7-e83f-46fa-b3eb-fc66178de01f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:05.941182 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26478b7-e83f-46fa-b3eb-fc66178de01f service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:05.941529 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26478b7-e83f-46fa-b3eb-fc66178de01f service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:05.941812 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26478b7-e83f-46fa-b3eb-fc66178de01f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:05.954818 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26478b7-e83f-46fa-b3eb-fc66178de01f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:05.956378 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26478b7-e83f-46fa-b3eb-fc66178de01f service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:05.970556 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1158/2298] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:05 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:23:06.163023 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc58696-81f6-4efa-ba52-5b51eca71868 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:06.164952 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-abc58696-81f6-4efa-ba52-5b51eca71868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:06.321154 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:06.327840 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc58696-81f6-4efa-ba52-5b51eca71868 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:06.341377 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde6e949-c0e4-4c3b-9e03-c54310bbaed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:06.343576 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6e949-c0e4-4c3b-9e03-c54310bbaed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:06.344536 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6e949-c0e4-4c3b-9e03-c54310bbaed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:06.344982 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6e949-c0e4-4c3b-9e03-c54310bbaed6 admin admin] PATH_INFO: `/v3/projects/374345273f37467f9b057268c2884843` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:06.348700 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde6e949-c0e4-4c3b-9e03-c54310bbaed6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=374345273f37467f9b057268c2884843)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:06.353655 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde6e949-c0e4-4c3b-9e03-c54310bbaed6 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:06.514907 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-bde6e949-c0e4-4c3b-9e03-c54310bbaed6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '374345273f37467f9b057268c2884843', 'request_id': 'req-bde6e949-c0e4-4c3b-9e03-c54310bbaed6'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:06.523728 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-bde6e949-c0e4-4c3b-9e03-c54310bbaed6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '374345273f37467f9b057268c2884843', 'request_id': 'req-bde6e949-c0e4-4c3b-9e03-c54310bbaed6'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:06.542803 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-bde6e949-c0e4-4c3b-9e03-c54310bbaed6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '374345273f37467f9b057268c2884843', 'request_id': 'req-bde6e949-c0e4-4c3b-9e03-c54310bbaed6'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:06.577439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-bde6e949-c0e4-4c3b-9e03-c54310bbaed6 admin admin] The token cache is being invalidate because project 374345273f37467f9b057268c2884843 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:23:06.579183 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-bde6e949-c0e4-4c3b-9e03-c54310bbaed6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:06.580134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-bde6e949-c0e4-4c3b-9e03-c54310bbaed6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:06.583713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1141/2299] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:23:06 2022] DELETE /identity/v3/projects/374345273f37467f9b057268c2884843 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:06.589162 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e1ffb-a9c3-40b9-95e8-da7f6dcc229c None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:06.616009 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e1ffb-a9c3-40b9-95e8-da7f6dcc229c None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:06.616400 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e1ffb-a9c3-40b9-95e8-da7f6dcc229c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:06.643982 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-833e1ffb-a9c3-40b9-95e8-da7f6dcc229c None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:06.645487 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-833e1ffb-a9c3-40b9-95e8-da7f6dcc229c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:06.768235 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1159/2300] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:23:06.777247 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab4995-434d-47bd-baa6-f577329e4a7a None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:06.777713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab4995-434d-47bd-baa6-f577329e4a7a None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:06.777790 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab4995-434d-47bd-baa6-f577329e4a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:06.803188 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-78ab4995-434d-47bd-baa6-f577329e4a7a None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:06.804419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-78ab4995-434d-47bd-baa6-f577329e4a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:06.835115 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1142/2301] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:23:06.836369 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e1ffb-a9c3-40b9-95e8-da7f6dcc229c None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:06.856069 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e1ffb-a9c3-40b9-95e8-da7f6dcc229c None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:06.866479 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c91d2af-4cc4-479e-a90d-68c961e5d73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:06.867229 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c91d2af-4cc4-479e-a90d-68c961e5d73b admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:06.867581 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c91d2af-4cc4-479e-a90d-68c961e5d73b admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:06.867932 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c91d2af-4cc4-479e-a90d-68c961e5d73b admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:06.868977 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c91d2af-4cc4-479e-a90d-68c961e5d73b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:23:06.869353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c91d2af-4cc4-479e-a90d-68c961e5d73b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:06.871154 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c91d2af-4cc4-479e-a90d-68c961e5d73b admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:06.874141 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:06.889871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1160/2302] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:23:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:23:06.890145 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab4995-434d-47bd-baa6-f577329e4a7a None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:06.915060 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab4995-434d-47bd-baa6-f577329e4a7a None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:06.928645 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00f6ac-349d-44df-8cc6-680d82ddd66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:06.929505 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00f6ac-349d-44df-8cc6-680d82ddd66d admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:06.932912 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00f6ac-349d-44df-8cc6-680d82ddd66d admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:06.953932 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00f6ac-349d-44df-8cc6-680d82ddd66d admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:06.955621 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed00f6ac-349d-44df-8cc6-680d82ddd66d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:06.959091 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed00f6ac-349d-44df-8cc6-680d82ddd66d admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:07.022378 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1143/2303] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:23:06 2022] POST /identity/v3/projects => generated 381 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:07.026438 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c91d2af-4cc4-479e-a90d-68c961e5d73b admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:07.041224 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c91d2af-4cc4-479e-a90d-68c961e5d73b admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:07.125339 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f42b57-73a0-41aa-81cf-acbaf5b9fff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:07.126327 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f42b57-73a0-41aa-81cf-acbaf5b9fff5 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:07.128546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f42b57-73a0-41aa-81cf-acbaf5b9fff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:07.128829 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f42b57-73a0-41aa-81cf-acbaf5b9fff5 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:07.129515 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f42b57-73a0-41aa-81cf-acbaf5b9fff5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:07.131472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f42b57-73a0-41aa-81cf-acbaf5b9fff5 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:07.169658 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1161/2304] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:23:07 2022] POST /identity/v3/users => generated 291 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:07.184693 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00f6ac-349d-44df-8cc6-680d82ddd66d admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:07.204672 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00f6ac-349d-44df-8cc6-680d82ddd66d admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:07.209847 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a62bec-cb05-4467-a15d-662f9c260f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:07.210454 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a62bec-cb05-4467-a15d-662f9c260f49 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:07.210736 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a62bec-cb05-4467-a15d-662f9c260f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:07.211301 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a62bec-cb05-4467-a15d-662f9c260f49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:07.212252 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a62bec-cb05-4467-a15d-662f9c260f49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:07.214825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a62bec-cb05-4467-a15d-662f9c260f49 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:07.234919 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1144/2305] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:07 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:07.236368 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f42b57-73a0-41aa-81cf-acbaf5b9fff5 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:07.248942 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f42b57-73a0-41aa-81cf-acbaf5b9fff5 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:07.253866 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92c0f0-baa5-4bd7-8a3e-19b1360ddb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:07.254528 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92c0f0-baa5-4bd7-8a3e-19b1360ddb6e admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:07.254811 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92c0f0-baa5-4bd7-8a3e-19b1360ddb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:07.255091 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92c0f0-baa5-4bd7-8a3e-19b1360ddb6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:07.255805 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d92c0f0-baa5-4bd7-8a3e-19b1360ddb6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:07.257458 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d92c0f0-baa5-4bd7-8a3e-19b1360ddb6e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:07.275386 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1162/2306] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:07.280804 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a62bec-cb05-4467-a15d-662f9c260f49 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:07.293401 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a62bec-cb05-4467-a15d-662f9c260f49 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:07.299380 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27589f-c7d7-4f52-a8b4-028abcc57756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:07.301229 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27589f-c7d7-4f52-a8b4-028abcc57756 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:07.301594 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27589f-c7d7-4f52-a8b4-028abcc57756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:07.301873 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27589f-c7d7-4f52-a8b4-028abcc57756 admin admin] PATH_INFO: `/v3/projects/a4c6bc5f81ac457dbb91c5bc1a8f0d61/users/937a7097c9d84402882f73082cfa408f/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:07.323315 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27589f-c7d7-4f52-a8b4-028abcc57756 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4c6bc5f81ac457dbb91c5bc1a8f0d61, user_id=937a7097c9d84402882f73082cfa408f, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:07.328370 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27589f-c7d7-4f52-a8b4-028abcc57756 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:07.351767 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1145/2307] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:23:07 2022] PUT /identity/v3/projects/a4c6bc5f81ac457dbb91c5bc1a8f0d61/users/937a7097c9d84402882f73082cfa408f/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:07.357723 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4caad-bb6b-4880-b319-5bfd4dec43fe None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:07.358036 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4caad-bb6b-4880-b319-5bfd4dec43fe None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:07.358324 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4caad-bb6b-4880-b319-5bfd4dec43fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:07.386062 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-a8c4caad-bb6b-4880-b319-5bfd4dec43fe None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:07.387665 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a8c4caad-bb6b-4880-b319-5bfd4dec43fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:07.490003 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1163/2308] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:07.502683 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27589f-c7d7-4f52-a8b4-028abcc57756 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:07.511311 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ab27589f-c7d7-4f52-a8b4-028abcc57756 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:07.590703 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27589f-c7d7-4f52-a8b4-028abcc57756 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:07.594358 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22a8b6-816a-427a-831e-64a0ee973e75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:07.594983 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22a8b6-816a-427a-831e-64a0ee973e75 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:07.595269 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22a8b6-816a-427a-831e-64a0ee973e75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:07.595537 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22a8b6-816a-427a-831e-64a0ee973e75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:07.605629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba22a8b6-816a-427a-831e-64a0ee973e75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:07.607159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba22a8b6-816a-427a-831e-64a0ee973e75 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:07.640960 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1146/2309] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:23:08.996781 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e2597-3f9f-4d00-bcfe-0e247b02b945 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:08.997264 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e2597-3f9f-4d00-bcfe-0e247b02b945 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:08.997453 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e2597-3f9f-4d00-bcfe-0e247b02b945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:09.050761 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-006e2597-3f9f-4d00-bcfe-0e247b02b945 None None] MFA Rules not processed for user `937a7097c9d84402882f73082cfa408f`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:09.052359 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-006e2597-3f9f-4d00-bcfe-0e247b02b945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:09.123524 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22a8b6-816a-427a-831e-64a0ee973e75 service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:09.125393 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ba22a8b6-816a-427a-831e-64a0ee973e75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:09.151268 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22a8b6-816a-427a-831e-64a0ee973e75 service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:09.156252 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e3b9fd-112c-42e9-bf5e-d9f53e74eb2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:09.156911 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3b9fd-112c-42e9-bf5e-d9f53e74eb2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:09.157195 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3b9fd-112c-42e9-bf5e-d9f53e74eb2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:09.157475 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3b9fd-112c-42e9-bf5e-d9f53e74eb2a admin admin] PATH_INFO: `/v3/users/ce13b58459f741d8a83c91bd1bbef0fd` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:09.158696 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e3b9fd-112c-42e9-bf5e-d9f53e74eb2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce13b58459f741d8a83c91bd1bbef0fd)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:09.160639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e3b9fd-112c-42e9-bf5e-d9f53e74eb2a admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:09.183130 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1164/2310] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:23:09.184126 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006e2597-3f9f-4d00-bcfe-0e247b02b945 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:09.186062 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-006e2597-3f9f-4d00-bcfe-0e247b02b945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:09.193483 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:09.201906 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:09.209117 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006e2597-3f9f-4d00-bcfe-0e247b02b945 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:09.216030 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e059e3-6489-4d08-a8a4-ccea2504e38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:09.216937 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e059e3-6489-4d08-a8a4-ccea2504e38c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:09.217942 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e059e3-6489-4d08-a8a4-ccea2504e38c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:09.218557 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e059e3-6489-4d08-a8a4-ccea2504e38c admin admin] PATH_INFO: `/v3/users/a430d7aa4dcf40e399fcb5d1f4bf3f76` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:09.220222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e059e3-6489-4d08-a8a4-ccea2504e38c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a430d7aa4dcf40e399fcb5d1f4bf3f76)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:09.228787 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e059e3-6489-4d08-a8a4-ccea2504e38c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:09.241677 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c4e3b9fd-112c-42e9-bf5e-d9f53e74eb2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce13b58459f741d8a83c91bd1bbef0fd'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:09.242305 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c4e3b9fd-112c-42e9-bf5e-d9f53e74eb2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce13b58459f741d8a83c91bd1bbef0fd'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:09.265610 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c4e3b9fd-112c-42e9-bf5e-d9f53e74eb2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce13b58459f741d8a83c91bd1bbef0fd'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:09.285364 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c4e3b9fd-112c-42e9-bf5e-d9f53e74eb2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce13b58459f741d8a83c91bd1bbef0fd'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:09.293397 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-30e059e3-6489-4d08-a8a4-ccea2504e38c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a430d7aa4dcf40e399fcb5d1f4bf3f76'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:09.293960 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c4e3b9fd-112c-42e9-bf5e-d9f53e74eb2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce13b58459f741d8a83c91bd1bbef0fd'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:09.294202 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-30e059e3-6489-4d08-a8a4-ccea2504e38c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a430d7aa4dcf40e399fcb5d1f4bf3f76'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:09.294748 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-30e059e3-6489-4d08-a8a4-ccea2504e38c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a430d7aa4dcf40e399fcb5d1f4bf3f76'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:09.301201 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1147/2311] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:23:09 2022] DELETE /identity/v3/users/ce13b58459f741d8a83c91bd1bbef0fd => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:23:09.302225 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e3b9fd-112c-42e9-bf5e-d9f53e74eb2a admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:09.303707 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c4e3b9fd-112c-42e9-bf5e-d9f53e74eb2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:09.312797 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-30e059e3-6489-4d08-a8a4-ccea2504e38c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a430d7aa4dcf40e399fcb5d1f4bf3f76'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:09.319611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-30e059e3-6489-4d08-a8a4-ccea2504e38c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a430d7aa4dcf40e399fcb5d1f4bf3f76'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:09.337464 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1165/2312] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:23:09 2022] DELETE /identity/v3/users/a430d7aa4dcf40e399fcb5d1f4bf3f76 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:09.338722 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa8f1d-8223-4a19-945e-85239d0ad075 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:09.339239 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa8f1d-8223-4a19-945e-85239d0ad075 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:09.339599 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa8f1d-8223-4a19-945e-85239d0ad075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:09.360573 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-5efa8f1d-8223-4a19-945e-85239d0ad075 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:09.361810 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5efa8f1d-8223-4a19-945e-85239d0ad075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:09.409275 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e3b9fd-112c-42e9-bf5e-d9f53e74eb2a admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:09.462410 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1166/2313] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:09.475600 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa8f1d-8223-4a19-945e-85239d0ad075 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:09.477386 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5efa8f1d-8223-4a19-945e-85239d0ad075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:09.497213 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0921c34-369d-4ced-82c5-b951470b3afc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:09.498241 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0921c34-369d-4ced-82c5-b951470b3afc service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:09.498530 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0921c34-369d-4ced-82c5-b951470b3afc service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:09.498814 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0921c34-369d-4ced-82c5-b951470b3afc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:09.500753 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-a0921c34-369d-4ced-82c5-b951470b3afc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:09.556332 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa8f1d-8223-4a19-945e-85239d0ad075 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:09.572160 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0695af44-79a8-4461-83ed-7ffe663d7cb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:09.572949 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695af44-79a8-4461-83ed-7ffe663d7cb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:09.573275 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695af44-79a8-4461-83ed-7ffe663d7cb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:09.573570 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695af44-79a8-4461-83ed-7ffe663d7cb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:09.591442 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0921c34-369d-4ced-82c5-b951470b3afc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:09.593309 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0921c34-369d-4ced-82c5-b951470b3afc service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:09.597093 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0695af44-79a8-4461-83ed-7ffe663d7cb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:09.598733 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0695af44-79a8-4461-83ed-7ffe663d7cb1 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:09.611314 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1167/2314] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:23:09.612179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0695af44-79a8-4461-83ed-7ffe663d7cb1 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:09.614031 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-0695af44-79a8-4461-83ed-7ffe663d7cb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:09.623068 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1148/2315] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:23:09.648214 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0695af44-79a8-4461-83ed-7ffe663d7cb1 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:09.670198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb43ddd-9fd7-4d40-aac3-5fd5cae24b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:09.671321 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb43ddd-9fd7-4d40-aac3-5fd5cae24b70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:09.671795 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb43ddd-9fd7-4d40-aac3-5fd5cae24b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:09.672602 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb43ddd-9fd7-4d40-aac3-5fd5cae24b70 admin admin] PATH_INFO: `/v3/projects/7c14d53ffc0d4d90b7cbfbb6b3b5a83b` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:09.673617 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb43ddd-9fd7-4d40-aac3-5fd5cae24b70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c14d53ffc0d4d90b7cbfbb6b3b5a83b)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:09.676070 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb43ddd-9fd7-4d40-aac3-5fd5cae24b70 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:09.695962 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:09.751150 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-2bb43ddd-9fd7-4d40-aac3-5fd5cae24b70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c14d53ffc0d4d90b7cbfbb6b3b5a83b', 'request_id': 'req-2bb43ddd-9fd7-4d40-aac3-5fd5cae24b70'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:09.762528 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-2bb43ddd-9fd7-4d40-aac3-5fd5cae24b70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c14d53ffc0d4d90b7cbfbb6b3b5a83b', 'request_id': 'req-2bb43ddd-9fd7-4d40-aac3-5fd5cae24b70'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:09.776648 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-2bb43ddd-9fd7-4d40-aac3-5fd5cae24b70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c14d53ffc0d4d90b7cbfbb6b3b5a83b', 'request_id': 'req-2bb43ddd-9fd7-4d40-aac3-5fd5cae24b70'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:09.796824 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-2bb43ddd-9fd7-4d40-aac3-5fd5cae24b70 admin admin] The token cache is being invalidate because project 7c14d53ffc0d4d90b7cbfbb6b3b5a83b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:23:09.796824 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-2bb43ddd-9fd7-4d40-aac3-5fd5cae24b70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:09.798482 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-2bb43ddd-9fd7-4d40-aac3-5fd5cae24b70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:09.806341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1168/2316] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:23:09 2022] DELETE /identity/v3/projects/7c14d53ffc0d4d90b7cbfbb6b3b5a83b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:09.814095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3344e9a-8323-43cb-9dbb-4199a53b8623 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:09.814380 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3344e9a-8323-43cb-9dbb-4199a53b8623 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:09.814502 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3344e9a-8323-43cb-9dbb-4199a53b8623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:09.863687 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-d3344e9a-8323-43cb-9dbb-4199a53b8623 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:09.865523 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d3344e9a-8323-43cb-9dbb-4199a53b8623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:09.872435 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:09.980458 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1149/2317] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:09.988887 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a01c67-fa03-435c-b735-21365e87b958 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:09.989179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a01c67-fa03-435c-b735-21365e87b958 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:09.989460 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a01c67-fa03-435c-b735-21365e87b958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:10.022360 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3344e9a-8323-43cb-9dbb-4199a53b8623 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:10.026900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d3344e9a-8323-43cb-9dbb-4199a53b8623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:10.038949 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-31a01c67-fa03-435c-b735-21365e87b958 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:10.045692 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-31a01c67-fa03-435c-b735-21365e87b958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:10.052014 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3344e9a-8323-43cb-9dbb-4199a53b8623 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:10.068334 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088de3b1-a254-488d-ba55-4354f1519331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:10.069011 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088de3b1-a254-488d-ba55-4354f1519331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:10.069289 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088de3b1-a254-488d-ba55-4354f1519331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:10.069566 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088de3b1-a254-488d-ba55-4354f1519331 admin admin] PATH_INFO: `/v3/projects/3c8862593dda4d5c9c8ffd075d470d84` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:10.076523 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088de3b1-a254-488d-ba55-4354f1519331 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c8862593dda4d5c9c8ffd075d470d84)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:10.078532 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088de3b1-a254-488d-ba55-4354f1519331 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:10.085736 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1169/2318] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:10.087455 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a01c67-fa03-435c-b735-21365e87b958 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:10.106021 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a01c67-fa03-435c-b735-21365e87b958 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:10.128009 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f632e8-8535-4776-8f45-5a421564500e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:10.128878 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f632e8-8535-4776-8f45-5a421564500e admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:10.129389 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f632e8-8535-4776-8f45-5a421564500e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:10.129888 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f632e8-8535-4776-8f45-5a421564500e admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:10.140708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f632e8-8535-4776-8f45-5a421564500e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:23:10.141137 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f632e8-8535-4776-8f45-5a421564500e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:10.149047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f632e8-8535-4776-8f45-5a421564500e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:10.177383 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1170/2319] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:23:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:23:10.185710 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f632e8-8535-4776-8f45-5a421564500e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:10.214687 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f632e8-8535-4776-8f45-5a421564500e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:10.242658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-088de3b1-a254-488d-ba55-4354f1519331 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c8862593dda4d5c9c8ffd075d470d84', 'request_id': 'req-088de3b1-a254-488d-ba55-4354f1519331'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:10.253186 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-088de3b1-a254-488d-ba55-4354f1519331 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c8862593dda4d5c9c8ffd075d470d84', 'request_id': 'req-088de3b1-a254-488d-ba55-4354f1519331'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:10.264827 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-088de3b1-a254-488d-ba55-4354f1519331 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c8862593dda4d5c9c8ffd075d470d84', 'request_id': 'req-088de3b1-a254-488d-ba55-4354f1519331'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:10.287580 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-088de3b1-a254-488d-ba55-4354f1519331 admin admin] The token cache is being invalidate because project 3c8862593dda4d5c9c8ffd075d470d84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:23:10.288012 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-088de3b1-a254-488d-ba55-4354f1519331 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:10.288775 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-088de3b1-a254-488d-ba55-4354f1519331 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:10.290870 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1150/2320] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:23:10 2022] DELETE /identity/v3/projects/3c8862593dda4d5c9c8ffd075d470d84 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:23:10.295214 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088de3b1-a254-488d-ba55-4354f1519331 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:10.297163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-088de3b1-a254-488d-ba55-4354f1519331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:10.317580 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b96228-4771-4ac2-a13a-fa291dd7607b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:10.319512 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b96228-4771-4ac2-a13a-fa291dd7607b admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:10.320069 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b96228-4771-4ac2-a13a-fa291dd7607b admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:10.320348 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b96228-4771-4ac2-a13a-fa291dd7607b admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:10.323291 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b96228-4771-4ac2-a13a-fa291dd7607b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:10.326008 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b96228-4771-4ac2-a13a-fa291dd7607b admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:10.373325 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1171/2321] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:23:10 2022] POST /identity/v3/projects => generated 389 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:10.375037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b96228-4771-4ac2-a13a-fa291dd7607b admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:10.377461 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-23b96228-4771-4ac2-a13a-fa291dd7607b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:10.378779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088de3b1-a254-488d-ba55-4354f1519331 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:10.423414 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:10.454369 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc0ab7-8429-4bba-b9ec-014e39084aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:10.458198 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc0ab7-8429-4bba-b9ec-014e39084aa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:10.458198 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc0ab7-8429-4bba-b9ec-014e39084aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:10.458198 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc0ab7-8429-4bba-b9ec-014e39084aa8 admin admin] PATH_INFO: `/v3/users/71323123b47143cda7fcb800d0a686b6` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:10.461645 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc0ab7-8429-4bba-b9ec-014e39084aa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71323123b47143cda7fcb800d0a686b6)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:10.466064 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc0ab7-8429-4bba-b9ec-014e39084aa8 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:10.480265 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b96228-4771-4ac2-a13a-fa291dd7607b admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:10.487647 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd70ebb-8348-4efa-910a-00831e3dcc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:10.488390 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd70ebb-8348-4efa-910a-00831e3dcc47 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:10.488782 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd70ebb-8348-4efa-910a-00831e3dcc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:10.489284 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd70ebb-8348-4efa-910a-00831e3dcc47 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:10.490068 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd70ebb-8348-4efa-910a-00831e3dcc47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:10.492092 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd70ebb-8348-4efa-910a-00831e3dcc47 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:10.521116 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:10.529348 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1172/2322] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:23:10 2022] POST /identity/v3/users => generated 295 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:10.533679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd70ebb-8348-4efa-910a-00831e3dcc47 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:10.546460 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd70ebb-8348-4efa-910a-00831e3dcc47 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:10.548003 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d9bc0ab7-8429-4bba-b9ec-014e39084aa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71323123b47143cda7fcb800d0a686b6'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:10.548695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d9bc0ab7-8429-4bba-b9ec-014e39084aa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71323123b47143cda7fcb800d0a686b6'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:10.551341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb88e741-8c4d-40c2-bfe4-cb50f6253041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:10.551966 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88e741-8c4d-40c2-bfe4-cb50f6253041 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:10.552249 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88e741-8c4d-40c2-bfe4-cb50f6253041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:10.552531 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88e741-8c4d-40c2-bfe4-cb50f6253041 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:10.553265 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb88e741-8c4d-40c2-bfe4-cb50f6253041 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:10.554861 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb88e741-8c4d-40c2-bfe4-cb50f6253041 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:10.556527 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d9bc0ab7-8429-4bba-b9ec-014e39084aa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71323123b47143cda7fcb800d0a686b6'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:10.566179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1173/2323] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 04:23:10.570356 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb88e741-8c4d-40c2-bfe4-cb50f6253041 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:10.570804 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d9bc0ab7-8429-4bba-b9ec-014e39084aa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71323123b47143cda7fcb800d0a686b6'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:10.578706 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d9bc0ab7-8429-4bba-b9ec-014e39084aa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71323123b47143cda7fcb800d0a686b6'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:10.581305 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb88e741-8c4d-40c2-bfe4-cb50f6253041 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:10.585074 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1151/2324] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:23:10 2022] DELETE /identity/v3/users/71323123b47143cda7fcb800d0a686b6 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:23:10.586060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60956bd-9859-4424-b142-2580a20bbacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:10.586664 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60956bd-9859-4424-b142-2580a20bbacf admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:10.586948 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60956bd-9859-4424-b142-2580a20bbacf admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:10.587223 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60956bd-9859-4424-b142-2580a20bbacf admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:10.587946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60956bd-9859-4424-b142-2580a20bbacf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:10.589588 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60956bd-9859-4424-b142-2580a20bbacf admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:10.601328 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1174/2325] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:10.605549 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc0ab7-8429-4bba-b9ec-014e39084aa8 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:10.607339 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d9bc0ab7-8429-4bba-b9ec-014e39084aa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:10.684068 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc0ab7-8429-4bba-b9ec-014e39084aa8 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:10.689715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ddfc1-6418-4fcb-b30c-e44ae792cdf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:10.690739 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ddfc1-6418-4fcb-b30c-e44ae792cdf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:10.691094 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ddfc1-6418-4fcb-b30c-e44ae792cdf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:10.691379 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ddfc1-6418-4fcb-b30c-e44ae792cdf7 admin admin] PATH_INFO: `/v3/projects/243e75013f5f49e88df72564ac0d2fe7/users/0eaa342a6bc84529a5d252c7a9a161ba/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:10.710552 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6ddfc1-6418-4fcb-b30c-e44ae792cdf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=243e75013f5f49e88df72564ac0d2fe7, user_id=0eaa342a6bc84529a5d252c7a9a161ba, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:10.714968 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6ddfc1-6418-4fcb-b30c-e44ae792cdf7 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:10.731866 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1152/2326] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:23:10 2022] PUT /identity/v3/projects/243e75013f5f49e88df72564ac0d2fe7/users/0eaa342a6bc84529a5d252c7a9a161ba/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:10.737104 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3105c5-eba2-4cbd-8472-01428d822132 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:10.737417 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3105c5-eba2-4cbd-8472-01428d822132 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:10.737734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3105c5-eba2-4cbd-8472-01428d822132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:10.762786 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-bb3105c5-eba2-4cbd-8472-01428d822132 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:10.764213 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-bb3105c5-eba2-4cbd-8472-01428d822132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:10.842927 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1175/2327] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:10.852445 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ddfc1-6418-4fcb-b30c-e44ae792cdf7 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:10.854322 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-8f6ddfc1-6418-4fcb-b30c-e44ae792cdf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:10.903089 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3105c5-eba2-4cbd-8472-01428d822132 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:10.904845 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-bb3105c5-eba2-4cbd-8472-01428d822132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:10.907784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:10.918435 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3105c5-eba2-4cbd-8472-01428d822132 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:10.923161 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc314be-f199-4ca5-8c75-9582dae5bd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:10.923815 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc314be-f199-4ca5-8c75-9582dae5bd7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:10.924101 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc314be-f199-4ca5-8c75-9582dae5bd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:10.924379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc314be-f199-4ca5-8c75-9582dae5bd7f admin admin] PATH_INFO: `/v3/projects/fde1f9ac91c4444294318b5cbc3f441a` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:10.925568 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc314be-f199-4ca5-8c75-9582dae5bd7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fde1f9ac91c4444294318b5cbc3f441a)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:10.927563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc314be-f199-4ca5-8c75-9582dae5bd7f admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:10.941012 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ddfc1-6418-4fcb-b30c-e44ae792cdf7 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:10.946423 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d826f-25db-42d7-a828-388615053773 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:10.947205 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d826f-25db-42d7-a828-388615053773 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:10.948965 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d826f-25db-42d7-a828-388615053773 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:10.949495 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d826f-25db-42d7-a828-388615053773 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:10.960408 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d826f-25db-42d7-a828-388615053773 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:10.961997 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d826f-25db-42d7-a828-388615053773 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:10.972468 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1153/2328] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 04:23:10.980945 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3cc314be-f199-4ca5-8c75-9582dae5bd7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fde1f9ac91c4444294318b5cbc3f441a', 'request_id': 'req-3cc314be-f199-4ca5-8c75-9582dae5bd7f'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:10.987206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3cc314be-f199-4ca5-8c75-9582dae5bd7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fde1f9ac91c4444294318b5cbc3f441a', 'request_id': 'req-3cc314be-f199-4ca5-8c75-9582dae5bd7f'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:10.993785 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3cc314be-f199-4ca5-8c75-9582dae5bd7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fde1f9ac91c4444294318b5cbc3f441a', 'request_id': 'req-3cc314be-f199-4ca5-8c75-9582dae5bd7f'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:11.008083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3cc314be-f199-4ca5-8c75-9582dae5bd7f admin admin] The token cache is being invalidate because project fde1f9ac91c4444294318b5cbc3f441a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:23:11.008381 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3cc314be-f199-4ca5-8c75-9582dae5bd7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:11.009068 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3cc314be-f199-4ca5-8c75-9582dae5bd7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:11.011186 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1176/2329] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:23:10 2022] DELETE /identity/v3/projects/fde1f9ac91c4444294318b5cbc3f441a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:11.016280 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb9b3f-d575-4adb-ab48-1432b2d4e44e None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:11.016569 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb9b3f-d575-4adb-ab48-1432b2d4e44e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:11.016846 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb9b3f-d575-4adb-ab48-1432b2d4e44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:11.043260 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-2bbb9b3f-d575-4adb-ab48-1432b2d4e44e None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:11.044809 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2bbb9b3f-d575-4adb-ab48-1432b2d4e44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:11.116714 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1154/2330] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:11.126548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4aa32-fad5-4120-938b-f356a047f89e None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:11.127124 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4aa32-fad5-4120-938b-f356a047f89e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:11.127124 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4aa32-fad5-4120-938b-f356a047f89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:11.153477 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-66b4aa32-fad5-4120-938b-f356a047f89e None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:11.154752 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-66b4aa32-fad5-4120-938b-f356a047f89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:11.162899 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1177/2331] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:11.167574 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb9b3f-d575-4adb-ab48-1432b2d4e44e None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:11.178548 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb9b3f-d575-4adb-ab48-1432b2d4e44e None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:11.183757 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549b594-2330-40e5-9f85-4a0d6f13ff90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:11.184387 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549b594-2330-40e5-9f85-4a0d6f13ff90 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:11.184671 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549b594-2330-40e5-9f85-4a0d6f13ff90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:11.184978 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549b594-2330-40e5-9f85-4a0d6f13ff90 admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:11.185699 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a549b594-2330-40e5-9f85-4a0d6f13ff90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:23:11.186085 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a549b594-2330-40e5-9f85-4a0d6f13ff90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:11.190318 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a549b594-2330-40e5-9f85-4a0d6f13ff90 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:11.207623 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1155/2332] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:23:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:23:11.209177 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b4aa32-fad5-4120-938b-f356a047f89e None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:11.219965 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b4aa32-fad5-4120-938b-f356a047f89e None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:11.224780 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9fe33a-9911-4c5a-a493-a12ac6dd0095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:11.225706 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9fe33a-9911-4c5a-a493-a12ac6dd0095 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:11.225706 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9fe33a-9911-4c5a-a493-a12ac6dd0095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:11.226002 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9fe33a-9911-4c5a-a493-a12ac6dd0095 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:11.226787 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9fe33a-9911-4c5a-a493-a12ac6dd0095 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:11.228752 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9fe33a-9911-4c5a-a493-a12ac6dd0095 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:11.271413 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1178/2333] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:23:11 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:11.272949 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549b594-2330-40e5-9f85-4a0d6f13ff90 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:11.296361 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549b594-2330-40e5-9f85-4a0d6f13ff90 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:11.344274 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fa1e7-8063-4679-9a18-ce70bc4bbbe6 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:11.344709 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fa1e7-8063-4679-9a18-ce70bc4bbbe6 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:11.345094 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fa1e7-8063-4679-9a18-ce70bc4bbbe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:11.380594 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-2e6fa1e7-8063-4679-9a18-ce70bc4bbbe6 None None] MFA Rules not processed for user `937a7097c9d84402882f73082cfa408f`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:11.384697 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2e6fa1e7-8063-4679-9a18-ce70bc4bbbe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:11.406633 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc46020c-5315-4925-a1e2-9a78e5b23912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:11.409196 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46020c-5315-4925-a1e2-9a78e5b23912 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:11.409396 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46020c-5315-4925-a1e2-9a78e5b23912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:11.409699 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46020c-5315-4925-a1e2-9a78e5b23912 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:11.410414 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc46020c-5315-4925-a1e2-9a78e5b23912 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:11.416091 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc46020c-5315-4925-a1e2-9a78e5b23912 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:11.478221 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:11.484784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1156/2334] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:23:11 2022] POST /identity/v3/users => generated 292 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:11.489369 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc46020c-5315-4925-a1e2-9a78e5b23912 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:11.503175 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc46020c-5315-4925-a1e2-9a78e5b23912 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:11.525840 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f043540-fc52-4f89-b5dc-fddc44eef729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:11.527405 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f043540-fc52-4f89-b5dc-fddc44eef729 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:11.528204 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f043540-fc52-4f89-b5dc-fddc44eef729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:11.528695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f043540-fc52-4f89-b5dc-fddc44eef729 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:11.529639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f043540-fc52-4f89-b5dc-fddc44eef729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:11.532260 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f043540-fc52-4f89-b5dc-fddc44eef729 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:11.562013 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1179/2335] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:11.562543 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1157/2336] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:11 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 04:23:11.567734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6fa1e7-8063-4679-9a18-ce70bc4bbbe6 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:11.592234 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6fa1e7-8063-4679-9a18-ce70bc4bbbe6 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:11.608516 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13988e6e-bdb7-4fc8-8dbb-e8da35882960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:11.609322 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13988e6e-bdb7-4fc8-8dbb-e8da35882960 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:11.609655 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13988e6e-bdb7-4fc8-8dbb-e8da35882960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:11.610161 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13988e6e-bdb7-4fc8-8dbb-e8da35882960 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:11.611689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13988e6e-bdb7-4fc8-8dbb-e8da35882960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:11.614889 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13988e6e-bdb7-4fc8-8dbb-e8da35882960 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:11.627836 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1180/2337] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:11 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:11.632306 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f043540-fc52-4f89-b5dc-fddc44eef729 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:11.645865 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f043540-fc52-4f89-b5dc-fddc44eef729 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:11.651098 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c21312-ca95-4970-a22c-6a47e3cf159d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:11.652481 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c21312-ca95-4970-a22c-6a47e3cf159d admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:11.652840 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c21312-ca95-4970-a22c-6a47e3cf159d admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:11.653126 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c21312-ca95-4970-a22c-6a47e3cf159d admin admin] PATH_INFO: `/v3/projects/f699b6e67abb4018882e7e401588c220/users/ec1371a4fa1148dfb5687a91b681f00a/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:11.686716 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c21312-ca95-4970-a22c-6a47e3cf159d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f699b6e67abb4018882e7e401588c220, user_id=ec1371a4fa1148dfb5687a91b681f00a, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:11.689505 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c21312-ca95-4970-a22c-6a47e3cf159d admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:11.719034 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1158/2338] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:23:11 2022] PUT /identity/v3/projects/f699b6e67abb4018882e7e401588c220/users/ec1371a4fa1148dfb5687a91b681f00a/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:11.727764 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e302a-4bd5-46d4-a41a-cebce58322b0 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:11.728052 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e302a-4bd5-46d4-a41a-cebce58322b0 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:11.728463 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e302a-4bd5-46d4-a41a-cebce58322b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:11.771253 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-dd6e302a-4bd5-46d4-a41a-cebce58322b0 None None] MFA Rules not processed for user `ec1371a4fa1148dfb5687a91b681f00a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:11.772590 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-dd6e302a-4bd5-46d4-a41a-cebce58322b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:11.875025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1181/2339] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:11.886724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c21312-ca95-4970-a22c-6a47e3cf159d admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:11.888640 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-81c21312-ca95-4970-a22c-6a47e3cf159d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:11.933158 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:11.981036 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c21312-ca95-4970-a22c-6a47e3cf159d admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:11.986998 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6ac24c-2515-4aa3-9b33-7e737243a7d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:11.987759 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ac24c-2515-4aa3-9b33-7e737243a7d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:11.988121 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ac24c-2515-4aa3-9b33-7e737243a7d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:11.988394 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ac24c-2515-4aa3-9b33-7e737243a7d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:11.998523 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6ac24c-2515-4aa3-9b33-7e737243a7d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:11.999973 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6ac24c-2515-4aa3-9b33-7e737243a7d7 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:12.009111 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1159/2340] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:23:12.032351 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e302a-4bd5-46d4-a41a-cebce58322b0 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:12.101175 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e302a-4bd5-46d4-a41a-cebce58322b0 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:12.107416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac96e7f-9448-4305-b50f-4d3177980fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:12.108067 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac96e7f-9448-4305-b50f-4d3177980fdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:12.108444 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac96e7f-9448-4305-b50f-4d3177980fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:12.108611 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac96e7f-9448-4305-b50f-4d3177980fdd admin admin] PATH_INFO: `/v3/users/ec1371a4fa1148dfb5687a91b681f00a` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:12.109779 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac96e7f-9448-4305-b50f-4d3177980fdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec1371a4fa1148dfb5687a91b681f00a)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:12.111801 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac96e7f-9448-4305-b50f-4d3177980fdd admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:12.216712 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3ac96e7f-9448-4305-b50f-4d3177980fdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec1371a4fa1148dfb5687a91b681f00a'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:12.218975 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3ac96e7f-9448-4305-b50f-4d3177980fdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec1371a4fa1148dfb5687a91b681f00a'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:12.219885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3ac96e7f-9448-4305-b50f-4d3177980fdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec1371a4fa1148dfb5687a91b681f00a'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:12.232924 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:12.239475 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3ac96e7f-9448-4305-b50f-4d3177980fdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec1371a4fa1148dfb5687a91b681f00a'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:12.247283 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3ac96e7f-9448-4305-b50f-4d3177980fdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec1371a4fa1148dfb5687a91b681f00a'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:12.269634 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1182/2341] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:23:12 2022] DELETE /identity/v3/users/ec1371a4fa1148dfb5687a91b681f00a => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:12.274177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a138faa-feeb-47ef-8b3b-94c9d7804945 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:12.274648 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a138faa-feeb-47ef-8b3b-94c9d7804945 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:12.274760 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a138faa-feeb-47ef-8b3b-94c9d7804945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:12.320211 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-6a138faa-feeb-47ef-8b3b-94c9d7804945 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:12.321419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-6a138faa-feeb-47ef-8b3b-94c9d7804945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:12.404850 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1160/2342] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:12.412231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac96e7f-9448-4305-b50f-4d3177980fdd admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:12.414629 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-3ac96e7f-9448-4305-b50f-4d3177980fdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:12.513828 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac96e7f-9448-4305-b50f-4d3177980fdd admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:12.519853 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f6c2e-2dd8-4f47-8ac2-5a8d314012f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:12.520734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f6c2e-2dd8-4f47-8ac2-5a8d314012f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:12.521168 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f6c2e-2dd8-4f47-8ac2-5a8d314012f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:12.521633 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f6c2e-2dd8-4f47-8ac2-5a8d314012f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:12.538499 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f6c2e-2dd8-4f47-8ac2-5a8d314012f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:12.540375 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f6c2e-2dd8-4f47-8ac2-5a8d314012f6 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:12.588454 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1183/2343] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:23:13.676291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a138faa-feeb-47ef-8b3b-94c9d7804945 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:13.678427 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-6a138faa-feeb-47ef-8b3b-94c9d7804945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:13.713642 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a138faa-feeb-47ef-8b3b-94c9d7804945 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:13.726748 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f7f76-6518-4154-8070-4659269217fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:13.727509 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f7f76-6518-4154-8070-4659269217fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:13.728227 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f7f76-6518-4154-8070-4659269217fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:13.729323 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f7f76-6518-4154-8070-4659269217fc admin admin] PATH_INFO: `/v3/projects/f699b6e67abb4018882e7e401588c220` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:13.730454 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f7f76-6518-4154-8070-4659269217fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f699b6e67abb4018882e7e401588c220)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:13.732644 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f7f76-6518-4154-8070-4659269217fc admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:13.827300 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-e38f7f76-6518-4154-8070-4659269217fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f699b6e67abb4018882e7e401588c220', 'request_id': 'req-e38f7f76-6518-4154-8070-4659269217fc'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:13.837409 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-e38f7f76-6518-4154-8070-4659269217fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f699b6e67abb4018882e7e401588c220', 'request_id': 'req-e38f7f76-6518-4154-8070-4659269217fc'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:13.845813 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-e38f7f76-6518-4154-8070-4659269217fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f699b6e67abb4018882e7e401588c220', 'request_id': 'req-e38f7f76-6518-4154-8070-4659269217fc'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:13.868651 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-e38f7f76-6518-4154-8070-4659269217fc admin admin] The token cache is being invalidate because project f699b6e67abb4018882e7e401588c220 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:23:13.868828 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-e38f7f76-6518-4154-8070-4659269217fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:13.870057 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-e38f7f76-6518-4154-8070-4659269217fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:13.872738 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1161/2344] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:23:13 2022] DELETE /identity/v3/projects/f699b6e67abb4018882e7e401588c220 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:13.877787 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643adf77-78c3-4172-9da2-2eb5f4aa4b5a None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:13.878263 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643adf77-78c3-4172-9da2-2eb5f4aa4b5a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:13.878705 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643adf77-78c3-4172-9da2-2eb5f4aa4b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:13.922292 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-643adf77-78c3-4172-9da2-2eb5f4aa4b5a None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:13.924304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-643adf77-78c3-4172-9da2-2eb5f4aa4b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:14.055293 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1184/2345] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:14.064175 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4bc64-b1d7-494b-8ab6-4e1804cd538c None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:14.064757 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4bc64-b1d7-494b-8ab6-4e1804cd538c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:14.065372 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4bc64-b1d7-494b-8ab6-4e1804cd538c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:14.109925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643adf77-78c3-4172-9da2-2eb5f4aa4b5a None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:14.113039 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-643adf77-78c3-4172-9da2-2eb5f4aa4b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:14.134663 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643adf77-78c3-4172-9da2-2eb5f4aa4b5a None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:14.136764 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-7ff4bc64-b1d7-494b-8ab6-4e1804cd538c None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:14.141366 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e67329-32d0-4bd3-bcd5-e6c45b3b93d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:14.142106 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e67329-32d0-4bd3-bcd5-e6c45b3b93d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:14.142420 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e67329-32d0-4bd3-bcd5-e6c45b3b93d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:14.142884 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-7ff4bc64-b1d7-494b-8ab6-4e1804cd538c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:14.143314 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e67329-32d0-4bd3-bcd5-e6c45b3b93d3 admin admin] PATH_INFO: `/v3/users/00cce5451f3c4221813843147399ad5e` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:14.144586 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e67329-32d0-4bd3-bcd5-e6c45b3b93d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00cce5451f3c4221813843147399ad5e)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:14.146863 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e67329-32d0-4bd3-bcd5-e6c45b3b93d3 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:14.173442 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1162/2346] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:14.174689 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4bc64-b1d7-494b-8ab6-4e1804cd538c None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:14.200994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4bc64-b1d7-494b-8ab6-4e1804cd538c None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:14.225404 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c2209-155f-4f3c-bb9d-2597803f4eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:14.226491 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c2209-155f-4f3c-bb9d-2597803f4eaf admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:14.226802 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c2209-155f-4f3c-bb9d-2597803f4eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:14.227075 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c2209-155f-4f3c-bb9d-2597803f4eaf admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:14.232611 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c2209-155f-4f3c-bb9d-2597803f4eaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:23:14.232974 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c2209-155f-4f3c-bb9d-2597803f4eaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:14.234725 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c2209-155f-4f3c-bb9d-2597803f4eaf admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:14.257096 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a2e67329-32d0-4bd3-bcd5-e6c45b3b93d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00cce5451f3c4221813843147399ad5e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:14.258586 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a2e67329-32d0-4bd3-bcd5-e6c45b3b93d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00cce5451f3c4221813843147399ad5e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:14.259834 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a2e67329-32d0-4bd3-bcd5-e6c45b3b93d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00cce5451f3c4221813843147399ad5e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:14.270719 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1163/2347] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:23:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:23:14.273587 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c2209-155f-4f3c-bb9d-2597803f4eaf admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:14.279350 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-1c0c2209-155f-4f3c-bb9d-2597803f4eaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:14.318413 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a2e67329-32d0-4bd3-bcd5-e6c45b3b93d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00cce5451f3c4221813843147399ad5e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:14.329108 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:14.333021 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c2209-155f-4f3c-bb9d-2597803f4eaf admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:14.334944 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a2e67329-32d0-4bd3-bcd5-e6c45b3b93d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00cce5451f3c4221813843147399ad5e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:14.343516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99209061-c0a5-4f8a-9311-ea71d7cfedd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:14.344291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99209061-c0a5-4f8a-9311-ea71d7cfedd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:14.344733 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99209061-c0a5-4f8a-9311-ea71d7cfedd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:14.345131 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99209061-c0a5-4f8a-9311-ea71d7cfedd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:14.345967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99209061-c0a5-4f8a-9311-ea71d7cfedd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:14.349142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99209061-c0a5-4f8a-9311-ea71d7cfedd3 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:14.375760 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1185/2348] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:23:14 2022] DELETE /identity/v3/users/00cce5451f3c4221813843147399ad5e => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:14.460184 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1164/2349] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:23:14 2022] POST /identity/v3/projects => generated 383 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:14.464893 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e67329-32d0-4bd3-bcd5-e6c45b3b93d3 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:14.625345 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e67329-32d0-4bd3-bcd5-e6c45b3b93d3 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:14.777950 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b93c1fc-e965-4858-bb11-561dfadd6c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:14.778927 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93c1fc-e965-4858-bb11-561dfadd6c46 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:14.779322 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93c1fc-e965-4858-bb11-561dfadd6c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:14.779711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93c1fc-e965-4858-bb11-561dfadd6c46 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:14.780503 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b93c1fc-e965-4858-bb11-561dfadd6c46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:14.782599 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b93c1fc-e965-4858-bb11-561dfadd6c46 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:14.807866 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1186/2350] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:23:14 2022] POST /identity/v3/users => generated 292 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:14.812081 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99209061-c0a5-4f8a-9311-ea71d7cfedd3 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:14.825369 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99209061-c0a5-4f8a-9311-ea71d7cfedd3 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:14.830990 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92497324-d2e5-4ff8-b2fb-eedd9f720926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:14.831847 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92497324-d2e5-4ff8-b2fb-eedd9f720926 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:14.832150 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92497324-d2e5-4ff8-b2fb-eedd9f720926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:14.832913 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92497324-d2e5-4ff8-b2fb-eedd9f720926 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:14.833337 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92497324-d2e5-4ff8-b2fb-eedd9f720926 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:14.835784 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92497324-d2e5-4ff8-b2fb-eedd9f720926 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:14.842540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:14.850887 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1165/2351] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:14.855952 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b93c1fc-e965-4858-bb11-561dfadd6c46 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:14.874819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b93c1fc-e965-4858-bb11-561dfadd6c46 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:14.888080 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dec3aa-0c6f-4073-8955-cdfaf22f12e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:14.888892 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dec3aa-0c6f-4073-8955-cdfaf22f12e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:14.889607 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dec3aa-0c6f-4073-8955-cdfaf22f12e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:14.890030 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dec3aa-0c6f-4073-8955-cdfaf22f12e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:14.890982 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dec3aa-0c6f-4073-8955-cdfaf22f12e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:14.892757 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dec3aa-0c6f-4073-8955-cdfaf22f12e3 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:14.912345 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1187/2352] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:14 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:14.917408 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92497324-d2e5-4ff8-b2fb-eedd9f720926 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:14.936066 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92497324-d2e5-4ff8-b2fb-eedd9f720926 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:14.965948 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e5a6fe-7999-473f-bbd8-9c2bd9a34ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:14.966706 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5a6fe-7999-473f-bbd8-9c2bd9a34ab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:14.966994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5a6fe-7999-473f-bbd8-9c2bd9a34ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:14.967278 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5a6fe-7999-473f-bbd8-9c2bd9a34ab5 admin admin] PATH_INFO: `/v3/projects/0bad534938024a90821cee2ad0ecdac3/users/1625927880d84fd88fc439e2379aac4d/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:15.011537 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e5a6fe-7999-473f-bbd8-9c2bd9a34ab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bad534938024a90821cee2ad0ecdac3, user_id=1625927880d84fd88fc439e2379aac4d, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:15.014286 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e5a6fe-7999-473f-bbd8-9c2bd9a34ab5 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:15.042390 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1166/2353] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:23:14 2022] PUT /identity/v3/projects/0bad534938024a90821cee2ad0ecdac3/users/1625927880d84fd88fc439e2379aac4d/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:15.053609 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775d61d-ec7f-4ba5-a5c6-44b12677a7aa None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:15.054206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775d61d-ec7f-4ba5-a5c6-44b12677a7aa None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:15.054588 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775d61d-ec7f-4ba5-a5c6-44b12677a7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:15.113247 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-d775d61d-ec7f-4ba5-a5c6-44b12677a7aa None None] MFA Rules not processed for user `1625927880d84fd88fc439e2379aac4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:15.114915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d775d61d-ec7f-4ba5-a5c6-44b12677a7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:15.247977 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1188/2354] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:15.264837 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e5a6fe-7999-473f-bbd8-9c2bd9a34ab5 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:15.269864 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-70e5a6fe-7999-473f-bbd8-9c2bd9a34ab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:15.341558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775d61d-ec7f-4ba5-a5c6-44b12677a7aa None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:15.343170 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d775d61d-ec7f-4ba5-a5c6-44b12677a7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:15.350913 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:15.398484 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e5a6fe-7999-473f-bbd8-9c2bd9a34ab5 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:15.404748 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f8cb1-8d35-4301-894e-c89fb9c58feb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:15.405620 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f8cb1-8d35-4301-894e-c89fb9c58feb service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:15.406206 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f8cb1-8d35-4301-894e-c89fb9c58feb service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:15.406634 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f8cb1-8d35-4301-894e-c89fb9c58feb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:15.421636 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f8cb1-8d35-4301-894e-c89fb9c58feb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:15.423892 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f8cb1-8d35-4301-894e-c89fb9c58feb service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:15.438934 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1167/2355] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:23:15.512155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775d61d-ec7f-4ba5-a5c6-44b12677a7aa None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:15.517595 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c674cd-126a-43ef-a864-688277b862a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:15.518458 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c674cd-126a-43ef-a864-688277b862a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:15.518755 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c674cd-126a-43ef-a864-688277b862a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:15.519039 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c674cd-126a-43ef-a864-688277b862a1 admin admin] PATH_INFO: `/v3/projects/367dd306bba440ccaff10e20352c5393` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:15.520769 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c674cd-126a-43ef-a864-688277b862a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=367dd306bba440ccaff10e20352c5393)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:15.522778 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c674cd-126a-43ef-a864-688277b862a1 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:15.602037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-07c674cd-126a-43ef-a864-688277b862a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '367dd306bba440ccaff10e20352c5393', 'request_id': 'req-07c674cd-126a-43ef-a864-688277b862a1'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:15.614359 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-07c674cd-126a-43ef-a864-688277b862a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '367dd306bba440ccaff10e20352c5393', 'request_id': 'req-07c674cd-126a-43ef-a864-688277b862a1'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:15.628485 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-07c674cd-126a-43ef-a864-688277b862a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '367dd306bba440ccaff10e20352c5393', 'request_id': 'req-07c674cd-126a-43ef-a864-688277b862a1'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:15.652030 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-07c674cd-126a-43ef-a864-688277b862a1 admin admin] The token cache is being invalidate because project 367dd306bba440ccaff10e20352c5393 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:23:15.652395 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-07c674cd-126a-43ef-a864-688277b862a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:15.653074 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-07c674cd-126a-43ef-a864-688277b862a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:15.655157 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1189/2356] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:23:15 2022] DELETE /identity/v3/projects/367dd306bba440ccaff10e20352c5393 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:23:15.658929 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f8cb1-8d35-4301-894e-c89fb9c58feb service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:15.675896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f8cb1-8d35-4301-894e-c89fb9c58feb service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:15.784597 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15488ca6-c04c-45c8-88d2-a68063869d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:15.784597 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15488ca6-c04c-45c8-88d2-a68063869d47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:15.785124 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15488ca6-c04c-45c8-88d2-a68063869d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:15.785613 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15488ca6-c04c-45c8-88d2-a68063869d47 admin admin] PATH_INFO: `/v3/users/eab1e50b964e497290bdbacc96dd4b4e` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:15.787022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15488ca6-c04c-45c8-88d2-a68063869d47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eab1e50b964e497290bdbacc96dd4b4e)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:15.789303 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15488ca6-c04c-45c8-88d2-a68063869d47 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:15.856331 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-15488ca6-c04c-45c8-88d2-a68063869d47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eab1e50b964e497290bdbacc96dd4b4e'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:15.857389 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-15488ca6-c04c-45c8-88d2-a68063869d47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eab1e50b964e497290bdbacc96dd4b4e'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:15.867294 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-15488ca6-c04c-45c8-88d2-a68063869d47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eab1e50b964e497290bdbacc96dd4b4e'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:15.880255 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-15488ca6-c04c-45c8-88d2-a68063869d47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eab1e50b964e497290bdbacc96dd4b4e'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:15.889627 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-15488ca6-c04c-45c8-88d2-a68063869d47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eab1e50b964e497290bdbacc96dd4b4e'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:15.891303 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:15.909757 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1168/2357] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:23:15 2022] DELETE /identity/v3/users/eab1e50b964e497290bdbacc96dd4b4e => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:16.160368 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c674cd-126a-43ef-a864-688277b862a1 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:16.161857 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-07c674cd-126a-43ef-a864-688277b862a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:16.188570 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c674cd-126a-43ef-a864-688277b862a1 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:16.258891 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b5b630-f57c-4fff-9fe7-dae67c5fe793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:16.259419 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5b630-f57c-4fff-9fe7-dae67c5fe793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:16.259704 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5b630-f57c-4fff-9fe7-dae67c5fe793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:16.260513 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5b630-f57c-4fff-9fe7-dae67c5fe793 admin admin] PATH_INFO: `/v3/projects/4c13002ef6634d16ac10f4b5444e1ec3` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:16.261815 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b5b630-f57c-4fff-9fe7-dae67c5fe793 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c13002ef6634d16ac10f4b5444e1ec3)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:16.264972 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b5b630-f57c-4fff-9fe7-dae67c5fe793 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:16.316016 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:16.331397 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d4b5b630-f57c-4fff-9fe7-dae67c5fe793 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c13002ef6634d16ac10f4b5444e1ec3', 'request_id': 'req-d4b5b630-f57c-4fff-9fe7-dae67c5fe793'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:16.338758 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d4b5b630-f57c-4fff-9fe7-dae67c5fe793 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c13002ef6634d16ac10f4b5444e1ec3', 'request_id': 'req-d4b5b630-f57c-4fff-9fe7-dae67c5fe793'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:16.345821 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d4b5b630-f57c-4fff-9fe7-dae67c5fe793 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c13002ef6634d16ac10f4b5444e1ec3', 'request_id': 'req-d4b5b630-f57c-4fff-9fe7-dae67c5fe793'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:16.360328 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d4b5b630-f57c-4fff-9fe7-dae67c5fe793 admin admin] The token cache is being invalidate because project 4c13002ef6634d16ac10f4b5444e1ec3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:23:16.360621 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d4b5b630-f57c-4fff-9fe7-dae67c5fe793 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:16.361311 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d4b5b630-f57c-4fff-9fe7-dae67c5fe793 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:16.363989 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1190/2358] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:23:16 2022] DELETE /identity/v3/projects/4c13002ef6634d16ac10f4b5444e1ec3 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:16.368733 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a4038-b562-4813-84a4-d72339db56ac None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:16.369821 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a4038-b562-4813-84a4-d72339db56ac None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:16.370393 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a4038-b562-4813-84a4-d72339db56ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:16.392326 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-092a4038-b562-4813-84a4-d72339db56ac None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:16.393666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-092a4038-b562-4813-84a4-d72339db56ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:16.479637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1169/2359] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:23:16.488863 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b296043-4757-4c1d-bd77-0b4cfa6c606a None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:16.489390 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b296043-4757-4c1d-bd77-0b4cfa6c606a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:16.490363 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b296043-4757-4c1d-bd77-0b4cfa6c606a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:16.518271 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-2b296043-4757-4c1d-bd77-0b4cfa6c606a None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:16.520276 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2b296043-4757-4c1d-bd77-0b4cfa6c606a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:16.532842 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1191/2360] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:16.536870 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a4038-b562-4813-84a4-d72339db56ac None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:16.560481 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a4038-b562-4813-84a4-d72339db56ac None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:16.568751 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8bcb51-2806-408c-be1c-484dac84a393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:16.569445 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8bcb51-2806-408c-be1c-484dac84a393 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:16.570122 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8bcb51-2806-408c-be1c-484dac84a393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:16.570451 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8bcb51-2806-408c-be1c-484dac84a393 admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:16.572176 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8bcb51-2806-408c-be1c-484dac84a393 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:23:16.573126 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8bcb51-2806-408c-be1c-484dac84a393 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:16.575057 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8bcb51-2806-408c-be1c-484dac84a393 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:16.593486 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1170/2361] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:23:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:23:16.595099 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b296043-4757-4c1d-bd77-0b4cfa6c606a None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:16.608521 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b296043-4757-4c1d-bd77-0b4cfa6c606a None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:16.617532 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1761eae-021f-402b-81ac-2e6b4c10f7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:16.618545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1761eae-021f-402b-81ac-2e6b4c10f7d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:16.618859 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1761eae-021f-402b-81ac-2e6b4c10f7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:16.619826 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1761eae-021f-402b-81ac-2e6b4c10f7d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:16.620787 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1761eae-021f-402b-81ac-2e6b4c10f7d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:16.622648 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1761eae-021f-402b-81ac-2e6b4c10f7d5 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:16.668128 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1192/2362] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:23:16 2022] POST /identity/v3/projects => generated 383 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:16.669927 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8bcb51-2806-408c-be1c-484dac84a393 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:16.689522 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8bcb51-2806-408c-be1c-484dac84a393 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:16.775162 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb2cd8-d79a-4c70-ab74-99f4f480e12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:16.775678 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb2cd8-d79a-4c70-ab74-99f4f480e12b admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:16.775958 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb2cd8-d79a-4c70-ab74-99f4f480e12b admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:16.776230 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb2cd8-d79a-4c70-ab74-99f4f480e12b admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:16.777024 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb2cd8-d79a-4c70-ab74-99f4f480e12b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:16.779701 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb2cd8-d79a-4c70-ab74-99f4f480e12b admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:16.808461 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1171/2363] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:23:16 2022] POST /identity/v3/users => generated 292 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:16.812487 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1761eae-021f-402b-81ac-2e6b4c10f7d5 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:16.831162 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1761eae-021f-402b-81ac-2e6b4c10f7d5 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:16.842703 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a18ee-6dc0-45cc-969d-9b50f8469b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:16.843951 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a18ee-6dc0-45cc-969d-9b50f8469b32 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:16.844936 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a18ee-6dc0-45cc-969d-9b50f8469b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:16.845458 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a18ee-6dc0-45cc-969d-9b50f8469b32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:16.846525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619a18ee-6dc0-45cc-969d-9b50f8469b32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:16.850111 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619a18ee-6dc0-45cc-969d-9b50f8469b32 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:16.869876 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1193/2364] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:16 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:16.870490 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb2cd8-d79a-4c70-ab74-99f4f480e12b admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:16.895708 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb2cd8-d79a-4c70-ab74-99f4f480e12b admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:16.906736 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190b118-b201-40a8-bb7b-a1f2ddef84cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:16.906736 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190b118-b201-40a8-bb7b-a1f2ddef84cb admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:16.906736 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190b118-b201-40a8-bb7b-a1f2ddef84cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:16.906736 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190b118-b201-40a8-bb7b-a1f2ddef84cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:16.907175 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1190b118-b201-40a8-bb7b-a1f2ddef84cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:16.907175 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1190b118-b201-40a8-bb7b-a1f2ddef84cb admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:16.918830 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1172/2365] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:16 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 04:23:16.923507 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a18ee-6dc0-45cc-969d-9b50f8469b32 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:16.935149 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a18ee-6dc0-45cc-969d-9b50f8469b32 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:16.941745 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999df6d4-b441-4249-86ea-35f3e8d264bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:16.942260 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999df6d4-b441-4249-86ea-35f3e8d264bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:16.942420 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999df6d4-b441-4249-86ea-35f3e8d264bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:16.942749 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999df6d4-b441-4249-86ea-35f3e8d264bf admin admin] PATH_INFO: `/v3/projects/46b3efcfaea84066921b0f43facb13fd/users/4148fe3256244b668db983453fb06d8c/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:16.966375 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999df6d4-b441-4249-86ea-35f3e8d264bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=46b3efcfaea84066921b0f43facb13fd, user_id=4148fe3256244b668db983453fb06d8c, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:16.969165 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999df6d4-b441-4249-86ea-35f3e8d264bf admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:16.986300 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1194/2366] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:23:16 2022] PUT /identity/v3/projects/46b3efcfaea84066921b0f43facb13fd/users/4148fe3256244b668db983453fb06d8c/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:23:16.997065 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea80746-4047-4d31-b360-3734f6128b23 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:16.997349 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea80746-4047-4d31-b360-3734f6128b23 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:16.997624 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea80746-4047-4d31-b360-3734f6128b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:17.027433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-fea80746-4047-4d31-b360-3734f6128b23 None None] MFA Rules not processed for user `4148fe3256244b668db983453fb06d8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:17.028806 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-fea80746-4047-4d31-b360-3734f6128b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:17.121357 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:17.148743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1173/2367] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:17.154157 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999df6d4-b441-4249-86ea-35f3e8d264bf admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:17.224263 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999df6d4-b441-4249-86ea-35f3e8d264bf admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:17.230288 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d228f0-5ad8-4735-aa91-bbfa3be2997d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:17.231037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d228f0-5ad8-4735-aa91-bbfa3be2997d admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:17.231390 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d228f0-5ad8-4735-aa91-bbfa3be2997d admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:17.232098 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d228f0-5ad8-4735-aa91-bbfa3be2997d admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:17.232480 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d228f0-5ad8-4735-aa91-bbfa3be2997d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:17.234614 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d228f0-5ad8-4735-aa91-bbfa3be2997d admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:17.266027 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1195/2368] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:23:17 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:17.270281 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea80746-4047-4d31-b360-3734f6128b23 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:17.282759 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea80746-4047-4d31-b360-3734f6128b23 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:17.352553 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2218d8-aa8a-4b29-ac92-2175a71e4abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:17.353622 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2218d8-aa8a-4b29-ac92-2175a71e4abf admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:17.354661 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2218d8-aa8a-4b29-ac92-2175a71e4abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:17.355254 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2218d8-aa8a-4b29-ac92-2175a71e4abf admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:17.356202 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2218d8-aa8a-4b29-ac92-2175a71e4abf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:17.358547 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2218d8-aa8a-4b29-ac92-2175a71e4abf admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:17.385573 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1174/2369] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:23:17 2022] POST /identity/v3/users => generated 299 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:17.389789 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d228f0-5ad8-4735-aa91-bbfa3be2997d admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:17.403212 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d228f0-5ad8-4735-aa91-bbfa3be2997d admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:17.419416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8eae17-fb3a-44fc-8bf2-ee201ed4237f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:17.419416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8eae17-fb3a-44fc-8bf2-ee201ed4237f admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:17.419416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8eae17-fb3a-44fc-8bf2-ee201ed4237f admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:17.419416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8eae17-fb3a-44fc-8bf2-ee201ed4237f admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:17.419718 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8eae17-fb3a-44fc-8bf2-ee201ed4237f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:17.423418 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8eae17-fb3a-44fc-8bf2-ee201ed4237f admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:17.434021 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1196/2370] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:17 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 04:23:17.437386 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2218d8-aa8a-4b29-ac92-2175a71e4abf admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:17.448707 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2218d8-aa8a-4b29-ac92-2175a71e4abf admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:17.460464 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63e09c-0656-48a9-ad27-a1523fa765ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:17.461162 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63e09c-0656-48a9-ad27-a1523fa765ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:17.461441 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63e09c-0656-48a9-ad27-a1523fa765ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:17.461741 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63e09c-0656-48a9-ad27-a1523fa765ce admin admin] PATH_INFO: `/v3/projects/506437f3e75e448fae9833b10a9d3e20/users/b949d9bf35f04abb9d4007a6c88140e5/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:17.486545 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63e09c-0656-48a9-ad27-a1523fa765ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=506437f3e75e448fae9833b10a9d3e20, user_id=b949d9bf35f04abb9d4007a6c88140e5, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:17.490346 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63e09c-0656-48a9-ad27-a1523fa765ce admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:17.521283 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1175/2371] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:23:17 2022] PUT /identity/v3/projects/506437f3e75e448fae9833b10a9d3e20/users/b949d9bf35f04abb9d4007a6c88140e5/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:23:17.521927 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8eae17-fb3a-44fc-8bf2-ee201ed4237f admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:17.533634 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8eae17-fb3a-44fc-8bf2-ee201ed4237f admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:17.597144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:17.649215 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78404c2a-1a40-4bd9-9856-5ac1eafadaf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:17.650305 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78404c2a-1a40-4bd9-9856-5ac1eafadaf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:17.651246 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78404c2a-1a40-4bd9-9856-5ac1eafadaf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:17.651620 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78404c2a-1a40-4bd9-9856-5ac1eafadaf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:17.652468 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78404c2a-1a40-4bd9-9856-5ac1eafadaf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:17.654225 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78404c2a-1a40-4bd9-9856-5ac1eafadaf5 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:17.666357 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1197/2372] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:17 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 04:23:17.671563 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63e09c-0656-48a9-ad27-a1523fa765ce admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:17.691703 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63e09c-0656-48a9-ad27-a1523fa765ce admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:17.704970 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa0dcb-5d54-4fb0-b179-6269c26bd559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:17.705672 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa0dcb-5d54-4fb0-b179-6269c26bd559 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:17.706544 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa0dcb-5d54-4fb0-b179-6269c26bd559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:17.706544 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa0dcb-5d54-4fb0-b179-6269c26bd559 admin admin] PATH_INFO: `/v3/projects/506437f3e75e448fae9833b10a9d3e20/users/b949d9bf35f04abb9d4007a6c88140e5/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:17.708338 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa0dcb-5d54-4fb0-b179-6269c26bd559 admin admin] RBAC: Authorizing `identity:create_grant(project_id=506437f3e75e448fae9833b10a9d3e20, user_id=b949d9bf35f04abb9d4007a6c88140e5, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:17.714959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa0dcb-5d54-4fb0-b179-6269c26bd559 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:17.738022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1176/2373] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:23:17 2022] PUT /identity/v3/projects/506437f3e75e448fae9833b10a9d3e20/users/b949d9bf35f04abb9d4007a6c88140e5/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:17.752711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7398b-87bf-4338-a586-701fb1e5f670 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:17.753000 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7398b-87bf-4338-a586-701fb1e5f670 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:17.753276 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7398b-87bf-4338-a586-701fb1e5f670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:17.794213 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-b4e7398b-87bf-4338-a586-701fb1e5f670 None None] MFA Rules not processed for user `b949d9bf35f04abb9d4007a6c88140e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:17.795452 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b4e7398b-87bf-4338-a586-701fb1e5f670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:17.900790 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1198/2374] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:17 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:17.917303 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa0dcb-5d54-4fb0-b179-6269c26bd559 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:17.919473 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e0aa0dcb-5d54-4fb0-b179-6269c26bd559 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:18.029749 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa0dcb-5d54-4fb0-b179-6269c26bd559 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:18.033591 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f06236-3aa7-4a99-94cb-6e11098e62a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:18.034266 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f06236-3aa7-4a99-94cb-6e11098e62a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:18.034542 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f06236-3aa7-4a99-94cb-6e11098e62a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:18.035017 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f06236-3aa7-4a99-94cb-6e11098e62a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:18.162687 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f06236-3aa7-4a99-94cb-6e11098e62a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:18.164200 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f06236-3aa7-4a99-94cb-6e11098e62a3 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:18.170015 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:18.189203 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1177/2375] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:23:18.565190 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b69614-085b-4d31-8004-741c2d6a15fc None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:18.565645 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b69614-085b-4d31-8004-741c2d6a15fc None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:18.565645 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b69614-085b-4d31-8004-741c2d6a15fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:18.616563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-40b69614-085b-4d31-8004-741c2d6a15fc None None] MFA Rules not processed for user `0eaa342a6bc84529a5d252c7a9a161ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:18.618646 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-40b69614-085b-4d31-8004-741c2d6a15fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:18.812589 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1199/2376] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:18.827889 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f06236-3aa7-4a99-94cb-6e11098e62a3 service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:18.829813 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-47f06236-3aa7-4a99-94cb-6e11098e62a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:18.920902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f06236-3aa7-4a99-94cb-6e11098e62a3 service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:18.956261 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d76f72-02f5-447c-ac2b-9ac4db4cc132 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:18.957657 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d76f72-02f5-447c-ac2b-9ac4db4cc132 service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:18.958783 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d76f72-02f5-447c-ac2b-9ac4db4cc132 service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:18.959766 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d76f72-02f5-447c-ac2b-9ac4db4cc132 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:18.981868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d76f72-02f5-447c-ac2b-9ac4db4cc132 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:18.983751 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d76f72-02f5-447c-ac2b-9ac4db4cc132 service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:18.995139 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1178/2377] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:23:20.953312 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc15d6-ac67-452e-838a-732b1e33bd7a None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:20.953909 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc15d6-ac67-452e-838a-732b1e33bd7a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:20.953909 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc15d6-ac67-452e-838a-732b1e33bd7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:20.981503 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-77dc15d6-ac67-452e-838a-732b1e33bd7a None None] MFA Rules not processed for user `0eaa342a6bc84529a5d252c7a9a161ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:20.983309 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-77dc15d6-ac67-452e-838a-732b1e33bd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:21.015279 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1200/2378] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:25.228052 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d76f72-02f5-447c-ac2b-9ac4db4cc132 service nova] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:25.239050 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d76f72-02f5-447c-ac2b-9ac4db4cc132 service nova] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:25.242904 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5565425-3b14-43f5-8ee1-49a8b5a884d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:25.243521 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5565425-3b14-43f5-8ee1-49a8b5a884d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:25.243807 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5565425-3b14-43f5-8ee1-49a8b5a884d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:25.244778 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5565425-3b14-43f5-8ee1-49a8b5a884d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:25.256308 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5565425-3b14-43f5-8ee1-49a8b5a884d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:25.257999 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5565425-3b14-43f5-8ee1-49a8b5a884d5 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:25.269075 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1179/2379] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:25 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:23:30.972175 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=82 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:32.556896 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc15d6-ac67-452e-838a-732b1e33bd7a None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:32.559076 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-77dc15d6-ac67-452e-838a-732b1e33bd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:32.641257 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc15d6-ac67-452e-838a-732b1e33bd7a None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:32.646693 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7415bf-a89e-4fbf-8f3f-5fc8cddfa224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:32.647881 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7415bf-a89e-4fbf-8f3f-5fc8cddfa224 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:32.648347 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7415bf-a89e-4fbf-8f3f-5fc8cddfa224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:32.648810 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7415bf-a89e-4fbf-8f3f-5fc8cddfa224 admin admin] PATH_INFO: `/v3/users/1625927880d84fd88fc439e2379aac4d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:32.650801 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7415bf-a89e-4fbf-8f3f-5fc8cddfa224 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1625927880d84fd88fc439e2379aac4d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:32.652706 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7415bf-a89e-4fbf-8f3f-5fc8cddfa224 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:32.704757 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fe7415bf-a89e-4fbf-8f3f-5fc8cddfa224 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1625927880d84fd88fc439e2379aac4d'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:32.705467 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fe7415bf-a89e-4fbf-8f3f-5fc8cddfa224 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1625927880d84fd88fc439e2379aac4d'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:32.706142 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fe7415bf-a89e-4fbf-8f3f-5fc8cddfa224 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1625927880d84fd88fc439e2379aac4d'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:32.720039 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fe7415bf-a89e-4fbf-8f3f-5fc8cddfa224 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1625927880d84fd88fc439e2379aac4d'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:32.727490 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fe7415bf-a89e-4fbf-8f3f-5fc8cddfa224 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1625927880d84fd88fc439e2379aac4d'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:32.739800 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1201/2380] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:23:32 2022] DELETE /identity/v3/users/1625927880d84fd88fc439e2379aac4d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:32.744570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a4ef9-e0e7-47d3-91a9-94bd08a1fd3d None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:32.745106 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a4ef9-e0e7-47d3-91a9-94bd08a1fd3d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:32.745106 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a4ef9-e0e7-47d3-91a9-94bd08a1fd3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:32.763998 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-359a4ef9-e0e7-47d3-91a9-94bd08a1fd3d None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:32.765185 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-359a4ef9-e0e7-47d3-91a9-94bd08a1fd3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:32.824173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1180/2381] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:32.833836 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7415bf-a89e-4fbf-8f3f-5fc8cddfa224 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:32.835395 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-fe7415bf-a89e-4fbf-8f3f-5fc8cddfa224 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:32.898098 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7415bf-a89e-4fbf-8f3f-5fc8cddfa224 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:32.902398 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2ef71-f2f1-4d95-90e0-f28c76645f71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:32.903021 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2ef71-f2f1-4d95-90e0-f28c76645f71 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:32.903304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2ef71-f2f1-4d95-90e0-f28c76645f71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:32.903580 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2ef71-f2f1-4d95-90e0-f28c76645f71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:32.912491 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c2ef71-f2f1-4d95-90e0-f28c76645f71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:32.913908 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c2ef71-f2f1-4d95-90e0-f28c76645f71 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:32.923872 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1202/2382] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:23:33.487508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a4ef9-e0e7-47d3-91a9-94bd08a1fd3d None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:33.489079 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-359a4ef9-e0e7-47d3-91a9-94bd08a1fd3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:33.502177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a4ef9-e0e7-47d3-91a9-94bd08a1fd3d None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:33.506790 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d56ffc-dcea-4ba6-96c7-5ec5de0edc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:33.507440 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d56ffc-dcea-4ba6-96c7-5ec5de0edc49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:33.507721 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d56ffc-dcea-4ba6-96c7-5ec5de0edc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:33.508001 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d56ffc-dcea-4ba6-96c7-5ec5de0edc49 admin admin] PATH_INFO: `/v3/projects/0bad534938024a90821cee2ad0ecdac3` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:33.509168 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d56ffc-dcea-4ba6-96c7-5ec5de0edc49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bad534938024a90821cee2ad0ecdac3)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:33.511725 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d56ffc-dcea-4ba6-96c7-5ec5de0edc49 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:33.569196 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c8d56ffc-dcea-4ba6-96c7-5ec5de0edc49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bad534938024a90821cee2ad0ecdac3', 'request_id': 'req-c8d56ffc-dcea-4ba6-96c7-5ec5de0edc49'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:33.577028 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c8d56ffc-dcea-4ba6-96c7-5ec5de0edc49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bad534938024a90821cee2ad0ecdac3', 'request_id': 'req-c8d56ffc-dcea-4ba6-96c7-5ec5de0edc49'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:33.582796 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c8d56ffc-dcea-4ba6-96c7-5ec5de0edc49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bad534938024a90821cee2ad0ecdac3', 'request_id': 'req-c8d56ffc-dcea-4ba6-96c7-5ec5de0edc49'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:33.586348 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:33.597998 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c8d56ffc-dcea-4ba6-96c7-5ec5de0edc49 admin admin] The token cache is being invalidate because project 0bad534938024a90821cee2ad0ecdac3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:23:33.598291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c8d56ffc-dcea-4ba6-96c7-5ec5de0edc49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:33.599027 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-c8d56ffc-dcea-4ba6-96c7-5ec5de0edc49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:33.600616 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1181/2383] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:23:33 2022] DELETE /identity/v3/projects/0bad534938024a90821cee2ad0ecdac3 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:33.605548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c10d9-be19-4e4d-ac88-d0e5b47e83ca None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:33.606027 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c10d9-be19-4e4d-ac88-d0e5b47e83ca None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:33.606132 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c10d9-be19-4e4d-ac88-d0e5b47e83ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:33.625814 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-ae1c10d9-be19-4e4d-ac88-d0e5b47e83ca None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:33.627060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-ae1c10d9-be19-4e4d-ac88-d0e5b47e83ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:33.644589 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:33.693844 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1203/2384] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:33.702838 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1006329-b8c7-4c01-8aa6-c856fa146523 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:33.703231 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1006329-b8c7-4c01-8aa6-c856fa146523 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:33.703542 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1006329-b8c7-4c01-8aa6-c856fa146523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:33.744239 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-b1006329-b8c7-4c01-8aa6-c856fa146523 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:33.745643 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b1006329-b8c7-4c01-8aa6-c856fa146523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:33.755099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1182/2385] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:23:33.760254 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1c10d9-be19-4e4d-ac88-d0e5b47e83ca None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:33.770563 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1c10d9-be19-4e4d-ac88-d0e5b47e83ca None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:33.775463 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561cb03-f805-4e34-ad5f-44ae076678aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:33.776192 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561cb03-f805-4e34-ad5f-44ae076678aa admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:33.776600 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561cb03-f805-4e34-ad5f-44ae076678aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:33.776884 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561cb03-f805-4e34-ad5f-44ae076678aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:33.777857 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5561cb03-f805-4e34-ad5f-44ae076678aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:23:33.778212 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5561cb03-f805-4e34-ad5f-44ae076678aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:33.779821 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5561cb03-f805-4e34-ad5f-44ae076678aa admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:33.792799 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1204/2386] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:23:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:23:33.799773 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1006329-b8c7-4c01-8aa6-c856fa146523 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:33.809635 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1006329-b8c7-4c01-8aa6-c856fa146523 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:33.814603 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a68a9-d6c1-4a00-ba7e-78077ceaff8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:33.815246 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a68a9-d6c1-4a00-ba7e-78077ceaff8e admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:33.815525 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a68a9-d6c1-4a00-ba7e-78077ceaff8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:33.815815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a68a9-d6c1-4a00-ba7e-78077ceaff8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:33.816505 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0a68a9-d6c1-4a00-ba7e-78077ceaff8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:33.818473 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0a68a9-d6c1-4a00-ba7e-78077ceaff8e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:33.845683 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1183/2387] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:23:33 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:33.849998 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561cb03-f805-4e34-ad5f-44ae076678aa admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:33.859898 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561cb03-f805-4e34-ad5f-44ae076678aa admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:33.934506 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c5e70-15ae-4aea-8e1e-39bb5964e50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:33.935135 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c5e70-15ae-4aea-8e1e-39bb5964e50b admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:33.935418 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c5e70-15ae-4aea-8e1e-39bb5964e50b admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:33.935705 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c5e70-15ae-4aea-8e1e-39bb5964e50b admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:33.936363 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c5e70-15ae-4aea-8e1e-39bb5964e50b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:33.938292 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c5e70-15ae-4aea-8e1e-39bb5964e50b admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:33.959023 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1205/2388] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:23:33 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:33.964315 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a68a9-d6c1-4a00-ba7e-78077ceaff8e admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:33.974576 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a68a9-d6c1-4a00-ba7e-78077ceaff8e admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:33.980389 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6bd0de-5e9f-48d6-acc7-d3b3100c1688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:33.981002 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6bd0de-5e9f-48d6-acc7-d3b3100c1688 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:33.981286 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6bd0de-5e9f-48d6-acc7-d3b3100c1688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:33.981562 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6bd0de-5e9f-48d6-acc7-d3b3100c1688 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:33.982299 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6bd0de-5e9f-48d6-acc7-d3b3100c1688 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:33.983895 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6bd0de-5e9f-48d6-acc7-d3b3100c1688 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:33.995225 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1184/2389] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:33.999375 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c5e70-15ae-4aea-8e1e-39bb5964e50b admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:34.010350 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c5e70-15ae-4aea-8e1e-39bb5964e50b admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:34.015122 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f4019-8674-4560-a0da-2e4564c802f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:34.015778 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f4019-8674-4560-a0da-2e4564c802f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:34.016064 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f4019-8674-4560-a0da-2e4564c802f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:34.016342 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f4019-8674-4560-a0da-2e4564c802f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:34.017050 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f4019-8674-4560-a0da-2e4564c802f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:34.019049 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f4019-8674-4560-a0da-2e4564c802f0 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:34.031121 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1206/2390] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:34.035417 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6bd0de-5e9f-48d6-acc7-d3b3100c1688 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:34.046529 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6bd0de-5e9f-48d6-acc7-d3b3100c1688 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:34.051130 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490e2231-6ba2-48df-b76a-2005348926b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:34.051788 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490e2231-6ba2-48df-b76a-2005348926b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:34.052071 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490e2231-6ba2-48df-b76a-2005348926b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:34.052348 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490e2231-6ba2-48df-b76a-2005348926b5 admin admin] PATH_INFO: `/v3/projects/af52d2221ddc4cb8aded94f078049f32/users/f4100e9823e24c6588dcba6601242610/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:34.067184 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490e2231-6ba2-48df-b76a-2005348926b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af52d2221ddc4cb8aded94f078049f32, user_id=f4100e9823e24c6588dcba6601242610, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:34.069826 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490e2231-6ba2-48df-b76a-2005348926b5 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:34.084369 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1185/2391] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:23:34 2022] PUT /identity/v3/projects/af52d2221ddc4cb8aded94f078049f32/users/f4100e9823e24c6588dcba6601242610/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:23:34.095470 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6d51e-11ef-4dfc-ae70-122a74975519 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:34.095747 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6d51e-11ef-4dfc-ae70-122a74975519 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:34.096029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6d51e-11ef-4dfc-ae70-122a74975519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:34.122693 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-8fd6d51e-11ef-4dfc-ae70-122a74975519 None None] MFA Rules not processed for user `f4100e9823e24c6588dcba6601242610`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:34.123913 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8fd6d51e-11ef-4dfc-ae70-122a74975519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:34.187473 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1207/2392] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:34.191960 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490e2231-6ba2-48df-b76a-2005348926b5 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:34.254887 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490e2231-6ba2-48df-b76a-2005348926b5 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:34.260252 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3a14d-4815-4b3f-9eab-71b6da41ccaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:34.260906 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3a14d-4815-4b3f-9eab-71b6da41ccaf admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:34.261185 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3a14d-4815-4b3f-9eab-71b6da41ccaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:34.261459 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3a14d-4815-4b3f-9eab-71b6da41ccaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:34.262165 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb3a14d-4815-4b3f-9eab-71b6da41ccaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:34.264117 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb3a14d-4815-4b3f-9eab-71b6da41ccaf admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:34.292292 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1186/2393] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:23:34 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:34.296339 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd6d51e-11ef-4dfc-ae70-122a74975519 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:34.307147 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd6d51e-11ef-4dfc-ae70-122a74975519 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:34.319403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:34.364860 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229cc24-6880-4b1f-a49a-6d14d9468173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:34.365482 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229cc24-6880-4b1f-a49a-6d14d9468173 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:34.365766 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229cc24-6880-4b1f-a49a-6d14d9468173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:34.366097 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229cc24-6880-4b1f-a49a-6d14d9468173 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:34.366732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4229cc24-6880-4b1f-a49a-6d14d9468173 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:34.368640 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4229cc24-6880-4b1f-a49a-6d14d9468173 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:34.389990 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1208/2394] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:23:34 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:34.394027 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3a14d-4815-4b3f-9eab-71b6da41ccaf admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:34.405134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3a14d-4815-4b3f-9eab-71b6da41ccaf admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:34.409951 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86d482-08a7-47cd-8cf4-3ce262a41f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:34.410598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86d482-08a7-47cd-8cf4-3ce262a41f82 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:34.410885 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86d482-08a7-47cd-8cf4-3ce262a41f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:34.411163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86d482-08a7-47cd-8cf4-3ce262a41f82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:34.411867 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86d482-08a7-47cd-8cf4-3ce262a41f82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:34.413476 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86d482-08a7-47cd-8cf4-3ce262a41f82 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:34.424882 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1187/2395] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:34.428895 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229cc24-6880-4b1f-a49a-6d14d9468173 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:34.438954 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229cc24-6880-4b1f-a49a-6d14d9468173 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:34.443642 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8284ae14-71e8-4cbe-94d6-2c94e71a79e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:34.444262 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284ae14-71e8-4cbe-94d6-2c94e71a79e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:34.444548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284ae14-71e8-4cbe-94d6-2c94e71a79e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:34.444827 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284ae14-71e8-4cbe-94d6-2c94e71a79e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:34.445791 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8284ae14-71e8-4cbe-94d6-2c94e71a79e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:34.447922 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8284ae14-71e8-4cbe-94d6-2c94e71a79e4 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:34.458803 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1209/2396] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:34.464297 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86d482-08a7-47cd-8cf4-3ce262a41f82 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:34.475253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86d482-08a7-47cd-8cf4-3ce262a41f82 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:34.480553 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c8619-c995-441b-a278-b5c5b3a67a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:34.481237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c8619-c995-441b-a278-b5c5b3a67a5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:34.481550 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c8619-c995-441b-a278-b5c5b3a67a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:34.481831 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c8619-c995-441b-a278-b5c5b3a67a5b admin admin] PATH_INFO: `/v3/projects/e433f0344c18453288e68a1420d09586/users/3553d5b59d7b4e1ca84e22cd995e5624/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:34.502488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3c8619-c995-441b-a278-b5c5b3a67a5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e433f0344c18453288e68a1420d09586, user_id=3553d5b59d7b4e1ca84e22cd995e5624, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:34.505179 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3c8619-c995-441b-a278-b5c5b3a67a5b admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:34.521594 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1188/2397] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:23:34 2022] PUT /identity/v3/projects/e433f0344c18453288e68a1420d09586/users/3553d5b59d7b4e1ca84e22cd995e5624/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:23:34.533946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23ce32-bda1-425f-b71f-cdfaaa55c187 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:34.534268 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23ce32-bda1-425f-b71f-cdfaaa55c187 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:34.534552 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23ce32-bda1-425f-b71f-cdfaaa55c187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:34.598979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-df23ce32-bda1-425f-b71f-cdfaaa55c187 None None] MFA Rules not processed for user `3553d5b59d7b4e1ca84e22cd995e5624`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:34.600299 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-df23ce32-bda1-425f-b71f-cdfaaa55c187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:34.677819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1210/2398] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:23:34.683593 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c8619-c995-441b-a278-b5c5b3a67a5b admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:34.732596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:34.753158 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c8619-c995-441b-a278-b5c5b3a67a5b admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:34.759661 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6d5f1-1991-44ec-900d-ca34528cc447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:34.760610 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6d5f1-1991-44ec-900d-ca34528cc447 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:34.761193 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6d5f1-1991-44ec-900d-ca34528cc447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:34.761488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6d5f1-1991-44ec-900d-ca34528cc447 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:34.762245 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d6d5f1-1991-44ec-900d-ca34528cc447 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:34.764757 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d6d5f1-1991-44ec-900d-ca34528cc447 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:34.808112 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1189/2399] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:23:34 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:34.809440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23ce32-bda1-425f-b71f-cdfaaa55c187 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:34.821840 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23ce32-bda1-425f-b71f-cdfaaa55c187 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:34.890530 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4675e09-6549-4bb5-b347-2a6f35dbbc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:34.891250 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4675e09-6549-4bb5-b347-2a6f35dbbc10 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:34.891622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4675e09-6549-4bb5-b347-2a6f35dbbc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:34.891962 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4675e09-6549-4bb5-b347-2a6f35dbbc10 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:34.892713 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4675e09-6549-4bb5-b347-2a6f35dbbc10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:34.894729 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4675e09-6549-4bb5-b347-2a6f35dbbc10 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:34.917265 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1211/2400] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:23:34 2022] POST /identity/v3/users => generated 294 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:23:34.921213 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6d5f1-1991-44ec-900d-ca34528cc447 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:34.937465 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6d5f1-1991-44ec-900d-ca34528cc447 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:34.945732 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d04c34-8f60-45cc-8899-1aef2fa73686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:34.946737 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d04c34-8f60-45cc-8899-1aef2fa73686 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:34.947854 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d04c34-8f60-45cc-8899-1aef2fa73686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:34.948310 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d04c34-8f60-45cc-8899-1aef2fa73686 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:34.949234 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d04c34-8f60-45cc-8899-1aef2fa73686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:34.951192 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d04c34-8f60-45cc-8899-1aef2fa73686 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:34.963378 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1190/2401] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:34 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:34.968083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4675e09-6549-4bb5-b347-2a6f35dbbc10 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:34.999545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4675e09-6549-4bb5-b347-2a6f35dbbc10 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:35.005035 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b725256-28be-48e1-a578-713bd0cec0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:35.005868 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b725256-28be-48e1-a578-713bd0cec0c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:35.006106 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b725256-28be-48e1-a578-713bd0cec0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:35.007036 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b725256-28be-48e1-a578-713bd0cec0c4 admin admin] PATH_INFO: `/v3/projects/3a36ecb9ca2b467da57bb3a3c6179709/users/bdde139da26f46aab25091f696594513/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:35.024234 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:35.031487 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b725256-28be-48e1-a578-713bd0cec0c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a36ecb9ca2b467da57bb3a3c6179709, user_id=bdde139da26f46aab25091f696594513, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:35.037051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b725256-28be-48e1-a578-713bd0cec0c4 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:35.066136 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d04c34-8f60-45cc-8899-1aef2fa73686 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:35.066631 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1212/2402] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:23:34 2022] PUT /identity/v3/projects/3a36ecb9ca2b467da57bb3a3c6179709/users/bdde139da26f46aab25091f696594513/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:35.078554 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d04c34-8f60-45cc-8899-1aef2fa73686 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:35.142341 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0119afab-6eba-4f2f-b2db-500fb54c8b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:35.142964 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119afab-6eba-4f2f-b2db-500fb54c8b23 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:35.143245 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119afab-6eba-4f2f-b2db-500fb54c8b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:35.143515 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119afab-6eba-4f2f-b2db-500fb54c8b23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:35.144212 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0119afab-6eba-4f2f-b2db-500fb54c8b23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:35.145850 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0119afab-6eba-4f2f-b2db-500fb54c8b23 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:35.157407 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1191/2403] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:35 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:35.162290 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b725256-28be-48e1-a578-713bd0cec0c4 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:35.177311 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b725256-28be-48e1-a578-713bd0cec0c4 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:35.183531 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3238f4-0692-498b-9523-fdd9c8704a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:35.184247 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3238f4-0692-498b-9523-fdd9c8704a7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:35.184529 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3238f4-0692-498b-9523-fdd9c8704a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:35.184872 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3238f4-0692-498b-9523-fdd9c8704a7e admin admin] PATH_INFO: `/v3/projects/3a36ecb9ca2b467da57bb3a3c6179709/users/bdde139da26f46aab25091f696594513/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:35.187134 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3238f4-0692-498b-9523-fdd9c8704a7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a36ecb9ca2b467da57bb3a3c6179709, user_id=bdde139da26f46aab25091f696594513, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:35.189761 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3238f4-0692-498b-9523-fdd9c8704a7e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:35.203005 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1213/2404] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:23:35 2022] PUT /identity/v3/projects/3a36ecb9ca2b467da57bb3a3c6179709/users/bdde139da26f46aab25091f696594513/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:35.214046 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12e94e-8b46-4465-95da-92666f5e4a56 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:35.214336 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12e94e-8b46-4465-95da-92666f5e4a56 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:35.214611 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12e94e-8b46-4465-95da-92666f5e4a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:35.242417 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-3e12e94e-8b46-4465-95da-92666f5e4a56 None None] MFA Rules not processed for user `bdde139da26f46aab25091f696594513`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:35.243626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3e12e94e-8b46-4465-95da-92666f5e4a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:35.317821 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1192/2405] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:35 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:35.330080 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3238f4-0692-498b-9523-fdd9c8704a7e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:35.331744 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-3e3238f4-0692-498b-9523-fdd9c8704a7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:35.397999 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3238f4-0692-498b-9523-fdd9c8704a7e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:35.403488 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023cee5e-4093-4234-a330-853366555db2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:35.404529 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cee5e-4093-4234-a330-853366555db2 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:35.405024 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cee5e-4093-4234-a330-853366555db2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:35.405499 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cee5e-4093-4234-a330-853366555db2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:35.463472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023cee5e-4093-4234-a330-853366555db2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:35.466607 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023cee5e-4093-4234-a330-853366555db2 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:35.482516 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1214/2406] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:23:37.187095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e12e94e-8b46-4465-95da-92666f5e4a56 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:37.188533 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3e12e94e-8b46-4465-95da-92666f5e4a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:37.203886 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023cee5e-4093-4234-a330-853366555db2 service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:37.212880 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023cee5e-4093-4234-a330-853366555db2 service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:37.216640 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe09448-8c7f-4605-bbe8-ff667ab75d39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:37.217255 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe09448-8c7f-4605-bbe8-ff667ab75d39 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:37.217533 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe09448-8c7f-4605-bbe8-ff667ab75d39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:37.217808 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe09448-8c7f-4605-bbe8-ff667ab75d39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:37.233563 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:37.257540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e12e94e-8b46-4465-95da-92666f5e4a56 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:37.262793 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2623e11c-b114-4ee0-8f14-53749fa764e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:37.263440 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2623e11c-b114-4ee0-8f14-53749fa764e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:37.263722 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2623e11c-b114-4ee0-8f14-53749fa764e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:37.264001 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2623e11c-b114-4ee0-8f14-53749fa764e9 admin admin] PATH_INFO: `/v3/users/4148fe3256244b668db983453fb06d8c` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:37.265353 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2623e11c-b114-4ee0-8f14-53749fa764e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4148fe3256244b668db983453fb06d8c)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:37.267274 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2623e11c-b114-4ee0-8f14-53749fa764e9 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:37.277344 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe09448-8c7f-4605-bbe8-ff667ab75d39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:37.279039 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe09448-8c7f-4605-bbe8-ff667ab75d39 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:37.293775 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1215/2407] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:23:37.327099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2623e11c-b114-4ee0-8f14-53749fa764e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4148fe3256244b668db983453fb06d8c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:37.327960 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2623e11c-b114-4ee0-8f14-53749fa764e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4148fe3256244b668db983453fb06d8c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:37.335496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2623e11c-b114-4ee0-8f14-53749fa764e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4148fe3256244b668db983453fb06d8c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:37.352490 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2623e11c-b114-4ee0-8f14-53749fa764e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4148fe3256244b668db983453fb06d8c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:37.360868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2623e11c-b114-4ee0-8f14-53749fa764e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4148fe3256244b668db983453fb06d8c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:37.369344 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1193/2408] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:23:37 2022] DELETE /identity/v3/users/4148fe3256244b668db983453fb06d8c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:37.374340 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87deece4-05b3-490d-8b88-9f6f22ea9efe None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:37.374747 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87deece4-05b3-490d-8b88-9f6f22ea9efe None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:37.375137 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87deece4-05b3-490d-8b88-9f6f22ea9efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:37.401430 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-87deece4-05b3-490d-8b88-9f6f22ea9efe None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:37.403479 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-87deece4-05b3-490d-8b88-9f6f22ea9efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:37.443853 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:37.500741 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1216/2409] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:37.510952 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2623e11c-b114-4ee0-8f14-53749fa764e9 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:37.512622 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2623e11c-b114-4ee0-8f14-53749fa764e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:37.569635 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2623e11c-b114-4ee0-8f14-53749fa764e9 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:37.573714 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b92c72-7e9d-4dd3-95ca-af75c6615503 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:37.574367 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b92c72-7e9d-4dd3-95ca-af75c6615503 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:37.574645 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b92c72-7e9d-4dd3-95ca-af75c6615503 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:37.574916 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b92c72-7e9d-4dd3-95ca-af75c6615503 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:37.584190 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b92c72-7e9d-4dd3-95ca-af75c6615503 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:37.585647 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b92c72-7e9d-4dd3-95ca-af75c6615503 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:37.595739 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1194/2410] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 04:23:37.875911 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87deece4-05b3-490d-8b88-9f6f22ea9efe None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:37.877711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-87deece4-05b3-490d-8b88-9f6f22ea9efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:37.890808 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87deece4-05b3-490d-8b88-9f6f22ea9efe None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:37.895477 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a056a91-749f-4c51-9625-0f8bd21bd911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:37.896133 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a056a91-749f-4c51-9625-0f8bd21bd911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:37.896415 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a056a91-749f-4c51-9625-0f8bd21bd911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:37.896692 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a056a91-749f-4c51-9625-0f8bd21bd911 admin admin] PATH_INFO: `/v3/projects/46b3efcfaea84066921b0f43facb13fd` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:37.897865 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a056a91-749f-4c51-9625-0f8bd21bd911 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46b3efcfaea84066921b0f43facb13fd)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:37.899810 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a056a91-749f-4c51-9625-0f8bd21bd911 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:37.948995 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-2a056a91-749f-4c51-9625-0f8bd21bd911 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46b3efcfaea84066921b0f43facb13fd', 'request_id': 'req-2a056a91-749f-4c51-9625-0f8bd21bd911'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:37.955133 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-2a056a91-749f-4c51-9625-0f8bd21bd911 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46b3efcfaea84066921b0f43facb13fd', 'request_id': 'req-2a056a91-749f-4c51-9625-0f8bd21bd911'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:37.960637 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-2a056a91-749f-4c51-9625-0f8bd21bd911 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46b3efcfaea84066921b0f43facb13fd', 'request_id': 'req-2a056a91-749f-4c51-9625-0f8bd21bd911'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:37.973417 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-2a056a91-749f-4c51-9625-0f8bd21bd911 admin admin] The token cache is being invalidate because project 46b3efcfaea84066921b0f43facb13fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:23:37.973701 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-2a056a91-749f-4c51-9625-0f8bd21bd911 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:37.974335 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-2a056a91-749f-4c51-9625-0f8bd21bd911 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:37.975842 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1217/2411] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:23:37 2022] DELETE /identity/v3/projects/46b3efcfaea84066921b0f43facb13fd => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:37.980435 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b92c72-7e9d-4dd3-95ca-af75c6615503 service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:37.989846 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b92c72-7e9d-4dd3-95ca-af75c6615503 service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:38.047541 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e1a52-39c3-45d1-9592-161d874940d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:38.048187 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e1a52-39c3-45d1-9592-161d874940d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:38.048469 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e1a52-39c3-45d1-9592-161d874940d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:38.048744 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e1a52-39c3-45d1-9592-161d874940d9 admin admin] PATH_INFO: `/v3/users/b949d9bf35f04abb9d4007a6c88140e5` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:38.049921 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e1a52-39c3-45d1-9592-161d874940d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b949d9bf35f04abb9d4007a6c88140e5)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:38.051859 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e1a52-39c3-45d1-9592-161d874940d9 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:38.053508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:38.106251 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0b5e1a52-39c3-45d1-9592-161d874940d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b949d9bf35f04abb9d4007a6c88140e5'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:38.106907 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0b5e1a52-39c3-45d1-9592-161d874940d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b949d9bf35f04abb9d4007a6c88140e5'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:38.114571 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0b5e1a52-39c3-45d1-9592-161d874940d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b949d9bf35f04abb9d4007a6c88140e5'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:38.129630 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0b5e1a52-39c3-45d1-9592-161d874940d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b949d9bf35f04abb9d4007a6c88140e5'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:38.137064 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0b5e1a52-39c3-45d1-9592-161d874940d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b949d9bf35f04abb9d4007a6c88140e5'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:38.143396 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1195/2412] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:23:37 2022] DELETE /identity/v3/users/b949d9bf35f04abb9d4007a6c88140e5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:38.432693 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a056a91-749f-4c51-9625-0f8bd21bd911 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:38.434202 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2a056a91-749f-4c51-9625-0f8bd21bd911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:38.449831 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a056a91-749f-4c51-9625-0f8bd21bd911 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:38.528772 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496d746-f969-45f0-a1e6-e1ff6d60477d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:38.529438 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496d746-f969-45f0-a1e6-e1ff6d60477d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:38.529725 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496d746-f969-45f0-a1e6-e1ff6d60477d admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:38.530035 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496d746-f969-45f0-a1e6-e1ff6d60477d admin admin] PATH_INFO: `/v3/projects/506437f3e75e448fae9833b10a9d3e20` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:38.531238 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7496d746-f969-45f0-a1e6-e1ff6d60477d admin admin] RBAC: Authorizing `identity:delete_project(project_id=506437f3e75e448fae9833b10a9d3e20)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:38.533242 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7496d746-f969-45f0-a1e6-e1ff6d60477d admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:38.584083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7496d746-f969-45f0-a1e6-e1ff6d60477d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '506437f3e75e448fae9833b10a9d3e20', 'request_id': 'req-7496d746-f969-45f0-a1e6-e1ff6d60477d'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:38.591689 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7496d746-f969-45f0-a1e6-e1ff6d60477d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '506437f3e75e448fae9833b10a9d3e20', 'request_id': 'req-7496d746-f969-45f0-a1e6-e1ff6d60477d'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:38.594224 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:38.598842 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7496d746-f969-45f0-a1e6-e1ff6d60477d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '506437f3e75e448fae9833b10a9d3e20', 'request_id': 'req-7496d746-f969-45f0-a1e6-e1ff6d60477d'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:38.617724 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7496d746-f969-45f0-a1e6-e1ff6d60477d admin admin] The token cache is being invalidate because project 506437f3e75e448fae9833b10a9d3e20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:23:38.618071 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7496d746-f969-45f0-a1e6-e1ff6d60477d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:38.618968 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7496d746-f969-45f0-a1e6-e1ff6d60477d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:38.621257 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1218/2413] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:23:38 2022] DELETE /identity/v3/projects/506437f3e75e448fae9833b10a9d3e20 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:38.627344 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ab3e5-26a3-4d33-a456-a56c8f648482 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:38.627906 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ab3e5-26a3-4d33-a456-a56c8f648482 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:38.627906 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ab3e5-26a3-4d33-a456-a56c8f648482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:38.648930 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-018ab3e5-26a3-4d33-a456-a56c8f648482 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:38.650939 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-018ab3e5-26a3-4d33-a456-a56c8f648482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:38.716279 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1196/2414] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:23:38.724864 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c864d8b-fab2-4326-9487-ceac56d30139 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:38.725165 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c864d8b-fab2-4326-9487-ceac56d30139 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:38.725440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c864d8b-fab2-4326-9487-ceac56d30139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:38.747289 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-0c864d8b-fab2-4326-9487-ceac56d30139 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:38.748597 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-0c864d8b-fab2-4326-9487-ceac56d30139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:38.756714 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1219/2415] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:38.761702 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ab3e5-26a3-4d33-a456-a56c8f648482 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:38.771534 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ab3e5-26a3-4d33-a456-a56c8f648482 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:38.776789 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5abecf-a919-430b-8b3e-ee9bfc7cc2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:38.777433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5abecf-a919-430b-8b3e-ee9bfc7cc2aa admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:38.777715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5abecf-a919-430b-8b3e-ee9bfc7cc2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:38.778012 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5abecf-a919-430b-8b3e-ee9bfc7cc2aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:38.778719 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5abecf-a919-430b-8b3e-ee9bfc7cc2aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:23:38.779025 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5abecf-a919-430b-8b3e-ee9bfc7cc2aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:38.780649 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5abecf-a919-430b-8b3e-ee9bfc7cc2aa admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:38.792015 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1197/2416] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:23:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 04:23:38.796379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c864d8b-fab2-4326-9487-ceac56d30139 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:38.805395 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c864d8b-fab2-4326-9487-ceac56d30139 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:38.810077 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7e4d6-64e2-423e-9955-5ebf411bba60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:38.810677 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7e4d6-64e2-423e-9955-5ebf411bba60 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:38.810952 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7e4d6-64e2-423e-9955-5ebf411bba60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:38.811222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7e4d6-64e2-423e-9955-5ebf411bba60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:38.811880 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f7e4d6-64e2-423e-9955-5ebf411bba60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:38.813772 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f7e4d6-64e2-423e-9955-5ebf411bba60 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:38.840947 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1220/2417] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:23:38 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:38.845072 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5abecf-a919-430b-8b3e-ee9bfc7cc2aa admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:38.855572 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5abecf-a919-430b-8b3e-ee9bfc7cc2aa admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:38.912802 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246a5ff-aa1a-4009-b564-a582404aebbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:38.913415 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246a5ff-aa1a-4009-b564-a582404aebbf admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:38.913697 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246a5ff-aa1a-4009-b564-a582404aebbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:38.913992 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246a5ff-aa1a-4009-b564-a582404aebbf admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:38.914676 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f246a5ff-aa1a-4009-b564-a582404aebbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:38.916986 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f246a5ff-aa1a-4009-b564-a582404aebbf admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:38.927597 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:38.940752 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1198/2418] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:23:38 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:38.945232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7e4d6-64e2-423e-9955-5ebf411bba60 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:38.956155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7e4d6-64e2-423e-9955-5ebf411bba60 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:38.961094 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14676c1c-019c-47f2-94e1-f23e6b40e5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:38.961644 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14676c1c-019c-47f2-94e1-f23e6b40e5d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:38.961921 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14676c1c-019c-47f2-94e1-f23e6b40e5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:38.962231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14676c1c-019c-47f2-94e1-f23e6b40e5d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:38.963811 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14676c1c-019c-47f2-94e1-f23e6b40e5d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:38.965393 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14676c1c-019c-47f2-94e1-f23e6b40e5d4 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:38.975958 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1221/2419] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:38.985382 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246a5ff-aa1a-4009-b564-a582404aebbf admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:38.996600 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246a5ff-aa1a-4009-b564-a582404aebbf admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:39.001203 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c9b0f3-6515-4e6d-95fb-e30113b91a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:39.001820 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9b0f3-6515-4e6d-95fb-e30113b91a13 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:39.002131 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9b0f3-6515-4e6d-95fb-e30113b91a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:39.002401 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9b0f3-6515-4e6d-95fb-e30113b91a13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:39.003099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c9b0f3-6515-4e6d-95fb-e30113b91a13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:39.004687 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c9b0f3-6515-4e6d-95fb-e30113b91a13 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:39.015869 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1199/2420] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:39.020297 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14676c1c-019c-47f2-94e1-f23e6b40e5d4 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:39.030482 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14676c1c-019c-47f2-94e1-f23e6b40e5d4 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:39.035137 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4798c987-14af-4a9e-9796-9d0f92fe7956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:39.035803 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4798c987-14af-4a9e-9796-9d0f92fe7956 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:39.036081 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4798c987-14af-4a9e-9796-9d0f92fe7956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:39.036356 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4798c987-14af-4a9e-9796-9d0f92fe7956 admin admin] PATH_INFO: `/v3/projects/f60af79582d4481babcb9aefb1afb657/users/00668826ceba4fcb8cb245a0cd271026/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:39.051670 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4798c987-14af-4a9e-9796-9d0f92fe7956 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f60af79582d4481babcb9aefb1afb657, user_id=00668826ceba4fcb8cb245a0cd271026, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:39.054372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4798c987-14af-4a9e-9796-9d0f92fe7956 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:39.067897 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1222/2421] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:23:39 2022] PUT /identity/v3/projects/f60af79582d4481babcb9aefb1afb657/users/00668826ceba4fcb8cb245a0cd271026/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:39.080865 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195dab76-6958-4fea-b6d2-7df2dc4b77a3 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:39.081156 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195dab76-6958-4fea-b6d2-7df2dc4b77a3 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:39.081477 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195dab76-6958-4fea-b6d2-7df2dc4b77a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:39.108526 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-195dab76-6958-4fea-b6d2-7df2dc4b77a3 None None] MFA Rules not processed for user `00668826ceba4fcb8cb245a0cd271026`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:39.109745 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-195dab76-6958-4fea-b6d2-7df2dc4b77a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:39.171869 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1200/2422] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:39.176338 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4798c987-14af-4a9e-9796-9d0f92fe7956 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:39.242383 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4798c987-14af-4a9e-9796-9d0f92fe7956 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:39.247955 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf387fb-c9bf-4ec8-861a-75f552e2eacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:39.248589 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf387fb-c9bf-4ec8-861a-75f552e2eacf admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:39.248877 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf387fb-c9bf-4ec8-861a-75f552e2eacf admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:39.249162 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf387fb-c9bf-4ec8-861a-75f552e2eacf admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:39.249826 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf387fb-c9bf-4ec8-861a-75f552e2eacf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:39.251779 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf387fb-c9bf-4ec8-861a-75f552e2eacf admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:39.285359 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1223/2423] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:23:39 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:23:39.289336 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195dab76-6958-4fea-b6d2-7df2dc4b77a3 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:39.304691 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195dab76-6958-4fea-b6d2-7df2dc4b77a3 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:39.363865 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae8fd6-749e-4ab4-bc12-3a8f3084dcf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:39.364486 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae8fd6-749e-4ab4-bc12-3a8f3084dcf0 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:39.364765 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae8fd6-749e-4ab4-bc12-3a8f3084dcf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:39.365038 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae8fd6-749e-4ab4-bc12-3a8f3084dcf0 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:39.365703 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae8fd6-749e-4ab4-bc12-3a8f3084dcf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:39.367657 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae8fd6-749e-4ab4-bc12-3a8f3084dcf0 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:39.395935 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1201/2424] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:23:39 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:23:39.396037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf387fb-c9bf-4ec8-861a-75f552e2eacf admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:39.404723 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf387fb-c9bf-4ec8-861a-75f552e2eacf admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:39.409345 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ded497-505a-415b-830b-dd3418f5dd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:39.409951 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ded497-505a-415b-830b-dd3418f5dd13 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:39.410254 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ded497-505a-415b-830b-dd3418f5dd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:39.410523 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ded497-505a-415b-830b-dd3418f5dd13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:39.411207 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ded497-505a-415b-830b-dd3418f5dd13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:39.412774 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ded497-505a-415b-830b-dd3418f5dd13 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:39.423762 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1224/2425] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:39.427535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae8fd6-749e-4ab4-bc12-3a8f3084dcf0 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:39.437915 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae8fd6-749e-4ab4-bc12-3a8f3084dcf0 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:39.442454 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eea93b-9fca-4bd5-995e-0e749c72b9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:39.443050 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eea93b-9fca-4bd5-995e-0e749c72b9a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:39.443329 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eea93b-9fca-4bd5-995e-0e749c72b9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:39.443600 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eea93b-9fca-4bd5-995e-0e749c72b9a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:39.444284 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eea93b-9fca-4bd5-995e-0e749c72b9a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:39.446149 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eea93b-9fca-4bd5-995e-0e749c72b9a5 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:39.456162 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1202/2426] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:39.460324 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ded497-505a-415b-830b-dd3418f5dd13 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:39.470836 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ded497-505a-415b-830b-dd3418f5dd13 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:39.475388 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b77bc-7252-4f1b-afea-6012fda7f0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:39.476047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b77bc-7252-4f1b-afea-6012fda7f0f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:39.476329 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b77bc-7252-4f1b-afea-6012fda7f0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:39.476606 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b77bc-7252-4f1b-afea-6012fda7f0f3 admin admin] PATH_INFO: `/v3/projects/7f919dc8ddbe444684d3aa849bf045aa/users/affcb20277b443ba98ddb4744f59c0e7/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:39.491832 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b77bc-7252-4f1b-afea-6012fda7f0f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f919dc8ddbe444684d3aa849bf045aa, user_id=affcb20277b443ba98ddb4744f59c0e7, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:39.495382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b77bc-7252-4f1b-afea-6012fda7f0f3 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:39.508887 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1225/2427] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:23:39 2022] PUT /identity/v3/projects/7f919dc8ddbe444684d3aa849bf045aa/users/affcb20277b443ba98ddb4744f59c0e7/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:39.519742 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980e776-b4f0-49af-a9b8-d3c0f403889c None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:39.520030 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980e776-b4f0-49af-a9b8-d3c0f403889c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:39.520322 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980e776-b4f0-49af-a9b8-d3c0f403889c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:39.549028 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-c980e776-b4f0-49af-a9b8-d3c0f403889c None None] MFA Rules not processed for user `affcb20277b443ba98ddb4744f59c0e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:39.550326 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c980e776-b4f0-49af-a9b8-d3c0f403889c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:39.602048 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:39.627291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1203/2428] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:39.632438 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b77bc-7252-4f1b-afea-6012fda7f0f3 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:39.698526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:39.776212 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b77bc-7252-4f1b-afea-6012fda7f0f3 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:39.793184 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f1c819-02ff-4379-b9dc-e73e69e8e307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:39.794520 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f1c819-02ff-4379-b9dc-e73e69e8e307 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:39.796990 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f1c819-02ff-4379-b9dc-e73e69e8e307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:39.797285 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f1c819-02ff-4379-b9dc-e73e69e8e307 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:39.799192 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f1c819-02ff-4379-b9dc-e73e69e8e307 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:39.803453 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f1c819-02ff-4379-b9dc-e73e69e8e307 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:39.863211 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1226/2429] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:23:39 2022] POST /identity/v3/projects => generated 383 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:39.867832 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980e776-b4f0-49af-a9b8-d3c0f403889c None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:39.886016 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980e776-b4f0-49af-a9b8-d3c0f403889c None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:39.933913 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f1c819-02ff-4379-b9dc-e73e69e8e307 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:39.935656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a8f1c819-02ff-4379-b9dc-e73e69e8e307 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:39.953817 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d78a87d-b3b5-46c5-93be-1c5565e06c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:39.954721 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d78a87d-b3b5-46c5-93be-1c5565e06c2a admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:39.954721 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d78a87d-b3b5-46c5-93be-1c5565e06c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:39.955012 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d78a87d-b3b5-46c5-93be-1c5565e06c2a admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:39.955674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d78a87d-b3b5-46c5-93be-1c5565e06c2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:39.957580 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d78a87d-b3b5-46c5-93be-1c5565e06c2a admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:39.980487 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1204/2430] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:23:39 2022] POST /identity/v3/users => generated 298 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:39.985260 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d78a87d-b3b5-46c5-93be-1c5565e06c2a admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:39.996945 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d78a87d-b3b5-46c5-93be-1c5565e06c2a admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:40.001408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f1c819-02ff-4379-b9dc-e73e69e8e307 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:40.004366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7321d6d4-a77e-4ae3-8280-8d7bf0a2389f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:40.005486 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321d6d4-a77e-4ae3-8280-8d7bf0a2389f admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:40.005939 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8bc852-ecb5-4eec-b238-40b75bb33d0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:40.006121 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321d6d4-a77e-4ae3-8280-8d7bf0a2389f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:40.006509 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321d6d4-a77e-4ae3-8280-8d7bf0a2389f admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:40.006610 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8bc852-ecb5-4eec-b238-40b75bb33d0b service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:40.006869 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8bc852-ecb5-4eec-b238-40b75bb33d0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:40.007150 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8bc852-ecb5-4eec-b238-40b75bb33d0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:40.007276 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7321d6d4-a77e-4ae3-8280-8d7bf0a2389f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:40.009006 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-dd8bc852-ecb5-4eec-b238-40b75bb33d0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:40.009154 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7321d6d4-a77e-4ae3-8280-8d7bf0a2389f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:40.024394 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1205/2431] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:40.026349 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7321d6d4-a77e-4ae3-8280-8d7bf0a2389f admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:40.042794 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7321d6d4-a77e-4ae3-8280-8d7bf0a2389f admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:40.048288 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d413a3-ff5c-4342-8488-b9256806f2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:40.049170 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d413a3-ff5c-4342-8488-b9256806f2c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:40.049452 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d413a3-ff5c-4342-8488-b9256806f2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:40.049729 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d413a3-ff5c-4342-8488-b9256806f2c1 admin admin] PATH_INFO: `/v3/projects/3a4cde7208c04b68af7984735be38fb2/users/b3d7f445420249f8b2967cab16327f96/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:40.066016 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d413a3-ff5c-4342-8488-b9256806f2c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a4cde7208c04b68af7984735be38fb2, user_id=b3d7f445420249f8b2967cab16327f96, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:40.068945 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d413a3-ff5c-4342-8488-b9256806f2c1 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:40.084794 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8bc852-ecb5-4eec-b238-40b75bb33d0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:40.086682 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8bc852-ecb5-4eec-b238-40b75bb33d0b service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:40.089284 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1206/2432] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:23:40 2022] PUT /identity/v3/projects/3a4cde7208c04b68af7984735be38fb2/users/b3d7f445420249f8b2967cab16327f96/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:23:40.093785 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d413a3-ff5c-4342-8488-b9256806f2c1 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:40.096921 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1227/2433] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:39 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 04:23:40.163922 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d413a3-ff5c-4342-8488-b9256806f2c1 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:40.169248 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d846f-9669-4af4-9599-e2ec7cccfcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:40.169880 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d846f-9669-4af4-9599-e2ec7cccfcf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:40.170198 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d846f-9669-4af4-9599-e2ec7cccfcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:40.170476 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d846f-9669-4af4-9599-e2ec7cccfcf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:40.171181 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d846f-9669-4af4-9599-e2ec7cccfcf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:40.172781 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d846f-9669-4af4-9599-e2ec7cccfcf9 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:40.184036 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1207/2434] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:40.188376 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8bc852-ecb5-4eec-b238-40b75bb33d0b service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:40.199756 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8bc852-ecb5-4eec-b238-40b75bb33d0b service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:40.204714 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a905a58a-30ba-4248-8b9b-b14bbc9ee476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:40.205546 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905a58a-30ba-4248-8b9b-b14bbc9ee476 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:40.205836 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905a58a-30ba-4248-8b9b-b14bbc9ee476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:40.206271 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905a58a-30ba-4248-8b9b-b14bbc9ee476 admin admin] PATH_INFO: `/v3/projects/3a4cde7208c04b68af7984735be38fb2/users/b3d7f445420249f8b2967cab16327f96/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:40.208648 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a905a58a-30ba-4248-8b9b-b14bbc9ee476 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a4cde7208c04b68af7984735be38fb2, user_id=b3d7f445420249f8b2967cab16327f96, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:40.212179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a905a58a-30ba-4248-8b9b-b14bbc9ee476 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:40.225298 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1228/2435] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:23:40 2022] PUT /identity/v3/projects/3a4cde7208c04b68af7984735be38fb2/users/b3d7f445420249f8b2967cab16327f96/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:40.235862 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff11c6d-3754-457a-868b-2bb487d7b4f0 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:40.236134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff11c6d-3754-457a-868b-2bb487d7b4f0 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:40.236407 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff11c6d-3754-457a-868b-2bb487d7b4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:40.266529 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-2ff11c6d-3754-457a-868b-2bb487d7b4f0 None None] MFA Rules not processed for user `b3d7f445420249f8b2967cab16327f96`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:40.267865 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2ff11c6d-3754-457a-868b-2bb487d7b4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:40.291889 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:40.360397 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1208/2436] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:40.373602 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a905a58a-30ba-4248-8b9b-b14bbc9ee476 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:40.435029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a905a58a-30ba-4248-8b9b-b14bbc9ee476 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:40.438858 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36e593-624c-4083-ac56-f1bf52646f04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:40.439478 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36e593-624c-4083-ac56-f1bf52646f04 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:40.439757 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36e593-624c-4083-ac56-f1bf52646f04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:40.440035 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36e593-624c-4083-ac56-f1bf52646f04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:40.445879 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:40.523548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb36e593-624c-4083-ac56-f1bf52646f04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:40.525026 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb36e593-624c-4083-ac56-f1bf52646f04 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:40.535975 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1229/2437] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:23:42.155497 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff11c6d-3754-457a-868b-2bb487d7b4f0 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:42.157197 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2ff11c6d-3754-457a-868b-2bb487d7b4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:42.177379 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff11c6d-3754-457a-868b-2bb487d7b4f0 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:42.181085 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ba5f3-404e-49a7-a970-0d94ed859f51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:42.181737 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ba5f3-404e-49a7-a970-0d94ed859f51 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:42.182517 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ba5f3-404e-49a7-a970-0d94ed859f51 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:42.182517 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ba5f3-404e-49a7-a970-0d94ed859f51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:42.247927 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ba5f3-404e-49a7-a970-0d94ed859f51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:42.249432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ba5f3-404e-49a7-a970-0d94ed859f51 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:42.259165 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1209/2438] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:23:42.643866 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36e593-624c-4083-ac56-f1bf52646f04 service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:42.647398 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-cb36e593-624c-4083-ac56-f1bf52646f04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:42.761474 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36e593-624c-4083-ac56-f1bf52646f04 service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:42.767620 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365bec1-225a-4421-94b8-a4eb4996a559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:42.768554 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365bec1-225a-4421-94b8-a4eb4996a559 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:42.769078 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365bec1-225a-4421-94b8-a4eb4996a559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:42.769598 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365bec1-225a-4421-94b8-a4eb4996a559 admin admin] PATH_INFO: `/v3/users/615ba049b1294439bdd1e01ee90a5dc4` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:42.773710 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a365bec1-225a-4421-94b8-a4eb4996a559 admin admin] RBAC: Authorizing `identity:delete_user(user_id=615ba049b1294439bdd1e01ee90a5dc4)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:42.776171 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a365bec1-225a-4421-94b8-a4eb4996a559 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:42.874976 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a365bec1-225a-4421-94b8-a4eb4996a559 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '615ba049b1294439bdd1e01ee90a5dc4'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:42.875499 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a365bec1-225a-4421-94b8-a4eb4996a559 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '615ba049b1294439bdd1e01ee90a5dc4'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:42.876218 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a365bec1-225a-4421-94b8-a4eb4996a559 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '615ba049b1294439bdd1e01ee90a5dc4'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:42.893649 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a365bec1-225a-4421-94b8-a4eb4996a559 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '615ba049b1294439bdd1e01ee90a5dc4'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:42.903146 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a365bec1-225a-4421-94b8-a4eb4996a559 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '615ba049b1294439bdd1e01ee90a5dc4'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:42.931417 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1230/2439] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:23:42 2022] DELETE /identity/v3/users/615ba049b1294439bdd1e01ee90a5dc4 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:42.934560 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f40aa-7cd8-4e0d-9ffc-95abed59e09e None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:42.934943 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f40aa-7cd8-4e0d-9ffc-95abed59e09e None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:42.935509 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f40aa-7cd8-4e0d-9ffc-95abed59e09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:42.988573 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-906f40aa-7cd8-4e0d-9ffc-95abed59e09e None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:42.990866 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-906f40aa-7cd8-4e0d-9ffc-95abed59e09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:43.088305 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365bec1-225a-4421-94b8-a4eb4996a559 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:43.090142 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a365bec1-225a-4421-94b8-a4eb4996a559 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:43.106809 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1210/2440] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:43.118005 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f40aa-7cd8-4e0d-9ffc-95abed59e09e None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:43.120585 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-906f40aa-7cd8-4e0d-9ffc-95abed59e09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:43.131533 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:43.173683 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365bec1-225a-4421-94b8-a4eb4996a559 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:43.177840 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c0f57-e566-4db6-8664-4fc73ce9adaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:43.178557 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c0f57-e566-4db6-8664-4fc73ce9adaa service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:43.178801 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c0f57-e566-4db6-8664-4fc73ce9adaa service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:43.179129 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c0f57-e566-4db6-8664-4fc73ce9adaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:43.182903 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-ff4c0f57-e566-4db6-8664-4fc73ce9adaa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:43.201802 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f40aa-7cd8-4e0d-9ffc-95abed59e09e None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:43.205855 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af82a495-4e8e-4d8f-9587-bfdfd68dc143 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:43.206701 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82a495-4e8e-4d8f-9587-bfdfd68dc143 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:43.207057 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82a495-4e8e-4d8f-9587-bfdfd68dc143 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:43.207449 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82a495-4e8e-4d8f-9587-bfdfd68dc143 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:43.218473 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af82a495-4e8e-4d8f-9587-bfdfd68dc143 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:43.220155 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af82a495-4e8e-4d8f-9587-bfdfd68dc143 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:43.226004 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=100 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:43.232540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1211/2441] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:23:43.304915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c0f57-e566-4db6-8664-4fc73ce9adaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:43.307339 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c0f57-e566-4db6-8664-4fc73ce9adaa service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:43.319075 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1231/2442] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:43 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 04:23:43.611711 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af82a495-4e8e-4d8f-9587-bfdfd68dc143 service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:43.614405 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-af82a495-4e8e-4d8f-9587-bfdfd68dc143 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:43.629440 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af82a495-4e8e-4d8f-9587-bfdfd68dc143 service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:43.634359 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc850b-bc24-40f1-9445-5b2fd102a560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:43.635383 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc850b-bc24-40f1-9445-5b2fd102a560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:43.635746 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc850b-bc24-40f1-9445-5b2fd102a560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:43.636102 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc850b-bc24-40f1-9445-5b2fd102a560 admin admin] PATH_INFO: `/v3/projects/18ae062d31a1483ea9b6bdc75ee93b20` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:43.637408 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dc850b-bc24-40f1-9445-5b2fd102a560 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18ae062d31a1483ea9b6bdc75ee93b20)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:43.639559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dc850b-bc24-40f1-9445-5b2fd102a560 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:43.693430 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-22dc850b-bc24-40f1-9445-5b2fd102a560 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18ae062d31a1483ea9b6bdc75ee93b20', 'request_id': 'req-22dc850b-bc24-40f1-9445-5b2fd102a560'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:43.701612 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-22dc850b-bc24-40f1-9445-5b2fd102a560 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18ae062d31a1483ea9b6bdc75ee93b20', 'request_id': 'req-22dc850b-bc24-40f1-9445-5b2fd102a560'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:43.709304 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-22dc850b-bc24-40f1-9445-5b2fd102a560 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18ae062d31a1483ea9b6bdc75ee93b20', 'request_id': 'req-22dc850b-bc24-40f1-9445-5b2fd102a560'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:43.724535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-22dc850b-bc24-40f1-9445-5b2fd102a560 admin admin] The token cache is being invalidate because project 18ae062d31a1483ea9b6bdc75ee93b20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:23:43.725009 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-22dc850b-bc24-40f1-9445-5b2fd102a560 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:43.725713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-22dc850b-bc24-40f1-9445-5b2fd102a560 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:43.727531 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1212/2443] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:23:43 2022] DELETE /identity/v3/projects/18ae062d31a1483ea9b6bdc75ee93b20 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:23:43.733013 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48e744-96ea-49ab-a54d-9ac33476465c None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:43.733524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48e744-96ea-49ab-a54d-9ac33476465c None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:43.733622 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48e744-96ea-49ab-a54d-9ac33476465c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:43.770737 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-7d48e744-96ea-49ab-a54d-9ac33476465c None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:43.772100 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7d48e744-96ea-49ab-a54d-9ac33476465c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:43.846589 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1232/2444] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:43.855382 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652bbf1-6935-4463-a688-55cd0f4b1209 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:43.855787 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652bbf1-6935-4463-a688-55cd0f4b1209 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:43.856103 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652bbf1-6935-4463-a688-55cd0f4b1209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:43.877966 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-5652bbf1-6935-4463-a688-55cd0f4b1209 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:43.879514 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-5652bbf1-6935-4463-a688-55cd0f4b1209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:43.892510 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1213/2445] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:43.894095 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48e744-96ea-49ab-a54d-9ac33476465c None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:43.905584 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48e744-96ea-49ab-a54d-9ac33476465c None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:43.912020 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dec406-4aa7-4f4c-993a-44688871c23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:43.912645 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dec406-4aa7-4f4c-993a-44688871c23d admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:43.912932 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dec406-4aa7-4f4c-993a-44688871c23d admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:43.913210 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dec406-4aa7-4f4c-993a-44688871c23d admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:43.913923 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dec406-4aa7-4f4c-993a-44688871c23d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:23:43.914255 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dec406-4aa7-4f4c-993a-44688871c23d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:43.916224 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dec406-4aa7-4f4c-993a-44688871c23d admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:43.931776 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1233/2446] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:23:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:23:43.933480 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5652bbf1-6935-4463-a688-55cd0f4b1209 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:43.943015 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5652bbf1-6935-4463-a688-55cd0f4b1209 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:43.948639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ba288-aa84-4179-af13-db05ba2d3d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:43.949261 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ba288-aa84-4179-af13-db05ba2d3d33 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:43.949566 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ba288-aa84-4179-af13-db05ba2d3d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:43.949843 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ba288-aa84-4179-af13-db05ba2d3d33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:43.950546 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ba288-aa84-4179-af13-db05ba2d3d33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:43.952448 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ba288-aa84-4179-af13-db05ba2d3d33 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:43.984519 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1214/2447] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:23:43 2022] POST /identity/v3/projects => generated 389 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:43.988871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dec406-4aa7-4f4c-993a-44688871c23d admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:43.998995 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dec406-4aa7-4f4c-993a-44688871c23d admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:44.100422 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2592d-a631-4942-bdc9-5c90d827fff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:44.100422 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2592d-a631-4942-bdc9-5c90d827fff7 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:44.100717 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2592d-a631-4942-bdc9-5c90d827fff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:44.105104 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2592d-a631-4942-bdc9-5c90d827fff7 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:44.105924 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2592d-a631-4942-bdc9-5c90d827fff7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:44.112163 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2592d-a631-4942-bdc9-5c90d827fff7 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:44.132496 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:44.156319 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1234/2448] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:23:43 2022] POST /identity/v3/users => generated 295 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:44.162183 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ba288-aa84-4179-af13-db05ba2d3d33 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:44.184353 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ba288-aa84-4179-af13-db05ba2d3d33 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:44.191958 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde13d4-05b0-458f-8ec0-a695ed39e31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:44.192693 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde13d4-05b0-458f-8ec0-a695ed39e31e admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:44.193345 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde13d4-05b0-458f-8ec0-a695ed39e31e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:44.193602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde13d4-05b0-458f-8ec0-a695ed39e31e admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:44.194343 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde13d4-05b0-458f-8ec0-a695ed39e31e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:44.196355 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde13d4-05b0-458f-8ec0-a695ed39e31e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:44.210496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1215/2449] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:44.211970 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2592d-a631-4942-bdc9-5c90d827fff7 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:44.231793 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2592d-a631-4942-bdc9-5c90d827fff7 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:44.237390 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd42e3-38ce-4414-808e-dd0b153d20c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:44.241270 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd42e3-38ce-4414-808e-dd0b153d20c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:44.241602 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd42e3-38ce-4414-808e-dd0b153d20c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:44.241894 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd42e3-38ce-4414-808e-dd0b153d20c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:44.242779 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd42e3-38ce-4414-808e-dd0b153d20c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:44.246004 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd42e3-38ce-4414-808e-dd0b153d20c5 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:44.272739 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1235/2450] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 04:23:44.275271 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde13d4-05b0-458f-8ec0-a695ed39e31e admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:44.288727 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde13d4-05b0-458f-8ec0-a695ed39e31e admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:44.294813 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb01b8-1731-4cfc-9007-1a6b20c83ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:44.295976 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb01b8-1731-4cfc-9007-1a6b20c83ed5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:44.296480 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb01b8-1731-4cfc-9007-1a6b20c83ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:44.296946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb01b8-1731-4cfc-9007-1a6b20c83ed5 admin admin] PATH_INFO: `/v3/projects/697391cefcd64122af7944252141cfc3/users/6a601cf1725e449f80eee24e95a757ea/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:44.315041 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb01b8-1731-4cfc-9007-1a6b20c83ed5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=697391cefcd64122af7944252141cfc3, user_id=6a601cf1725e449f80eee24e95a757ea, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:44.318509 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb01b8-1731-4cfc-9007-1a6b20c83ed5 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:44.345062 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1216/2451] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:23:44 2022] PUT /identity/v3/projects/697391cefcd64122af7944252141cfc3/users/6a601cf1725e449f80eee24e95a757ea/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:44.356610 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ad43b-494c-47b9-b9b0-33728faed681 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:44.356905 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ad43b-494c-47b9-b9b0-33728faed681 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:44.357188 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ad43b-494c-47b9-b9b0-33728faed681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:44.394147 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-104ad43b-494c-47b9-b9b0-33728faed681 None None] MFA Rules not processed for user `6a601cf1725e449f80eee24e95a757ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:44.395779 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-104ad43b-494c-47b9-b9b0-33728faed681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:44.503124 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb01b8-1731-4cfc-9007-1a6b20c83ed5 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:44.504937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-edfb01b8-1731-4cfc-9007-1a6b20c83ed5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:44.520986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1236/2452] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:44.536915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ad43b-494c-47b9-b9b0-33728faed681 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:44.541417 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-104ad43b-494c-47b9-b9b0-33728faed681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:44.612926 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb01b8-1731-4cfc-9007-1a6b20c83ed5 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:44.621514 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e71b017-77cc-4c3a-99ac-7dcb6c1c7b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:44.622589 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71b017-77cc-4c3a-99ac-7dcb6c1c7b00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:44.623056 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71b017-77cc-4c3a-99ac-7dcb6c1c7b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:44.623551 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71b017-77cc-4c3a-99ac-7dcb6c1c7b00 admin admin] PATH_INFO: `/v3/users/f4100e9823e24c6588dcba6601242610` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:44.625449 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e71b017-77cc-4c3a-99ac-7dcb6c1c7b00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4100e9823e24c6588dcba6601242610)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:44.628623 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e71b017-77cc-4c3a-99ac-7dcb6c1c7b00 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:44.660986 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:44.692702 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ad43b-494c-47b9-b9b0-33728faed681 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:44.699000 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16b38d-4b54-446b-90c4-f3a3d7b20e6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:44.699960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16b38d-4b54-446b-90c4-f3a3d7b20e6d service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:44.700243 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16b38d-4b54-446b-90c4-f3a3d7b20e6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:44.700573 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16b38d-4b54-446b-90c4-f3a3d7b20e6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:44.711858 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4e71b017-77cc-4c3a-99ac-7dcb6c1c7b00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4100e9823e24c6588dcba6601242610'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:44.718991 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4e71b017-77cc-4c3a-99ac-7dcb6c1c7b00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4100e9823e24c6588dcba6601242610'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:44.722643 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16b38d-4b54-446b-90c4-f3a3d7b20e6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:44.724635 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16b38d-4b54-446b-90c4-f3a3d7b20e6d service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:44.735530 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4e71b017-77cc-4c3a-99ac-7dcb6c1c7b00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4100e9823e24c6588dcba6601242610'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:44.748363 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1237/2453] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:23:44.768304 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4e71b017-77cc-4c3a-99ac-7dcb6c1c7b00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4100e9823e24c6588dcba6601242610'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:44.780200 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4e71b017-77cc-4c3a-99ac-7dcb6c1c7b00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4100e9823e24c6588dcba6601242610'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:44.791953 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1217/2454] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:23:44 2022] DELETE /identity/v3/users/f4100e9823e24c6588dcba6601242610 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:23:44.796385 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5d02f-72ab-424e-91ad-feb60d3a6efe None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:44.797088 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5d02f-72ab-424e-91ad-feb60d3a6efe None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:44.797599 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5d02f-72ab-424e-91ad-feb60d3a6efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:44.829895 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-b4a5d02f-72ab-424e-91ad-feb60d3a6efe None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:44.834152 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b4a5d02f-72ab-424e-91ad-feb60d3a6efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:44.982594 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1238/2455] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:44.992774 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e71b017-77cc-4c3a-99ac-7dcb6c1c7b00 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:44.995173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-4e71b017-77cc-4c3a-99ac-7dcb6c1c7b00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:45.093396 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e71b017-77cc-4c3a-99ac-7dcb6c1c7b00 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:45.098200 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609bf12-2926-4bc2-8123-e3701d96a93f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:45.099519 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609bf12-2926-4bc2-8123-e3701d96a93f service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:45.099961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609bf12-2926-4bc2-8123-e3701d96a93f service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:45.100596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609bf12-2926-4bc2-8123-e3701d96a93f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:45.141864 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9609bf12-2926-4bc2-8123-e3701d96a93f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:45.144535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9609bf12-2926-4bc2-8123-e3701d96a93f service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:45.165255 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1218/2456] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:23:45.490261 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a5d02f-72ab-424e-91ad-feb60d3a6efe None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:45.492668 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b4a5d02f-72ab-424e-91ad-feb60d3a6efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:45.509105 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a5d02f-72ab-424e-91ad-feb60d3a6efe None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:45.514220 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc1fa2-71bf-4cf2-841c-c37ea9bfcd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:45.514870 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc1fa2-71bf-4cf2-841c-c37ea9bfcd83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:45.515284 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc1fa2-71bf-4cf2-841c-c37ea9bfcd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:45.515592 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc1fa2-71bf-4cf2-841c-c37ea9bfcd83 admin admin] PATH_INFO: `/v3/users/16a4f447415341e49549de51a12645b0` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:45.516766 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc1fa2-71bf-4cf2-841c-c37ea9bfcd83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16a4f447415341e49549de51a12645b0)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:45.518737 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc1fa2-71bf-4cf2-841c-c37ea9bfcd83 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:45.532879 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:45.578405 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a3cc1fa2-71bf-4cf2-841c-c37ea9bfcd83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16a4f447415341e49549de51a12645b0'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:45.579105 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a3cc1fa2-71bf-4cf2-841c-c37ea9bfcd83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16a4f447415341e49549de51a12645b0'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:45.579728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a3cc1fa2-71bf-4cf2-841c-c37ea9bfcd83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16a4f447415341e49549de51a12645b0'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:45.608680 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a3cc1fa2-71bf-4cf2-841c-c37ea9bfcd83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16a4f447415341e49549de51a12645b0'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:45.609780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609bf12-2926-4bc2-8123-e3701d96a93f service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:45.611328 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9609bf12-2926-4bc2-8123-e3701d96a93f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:45.637550 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609bf12-2926-4bc2-8123-e3701d96a93f service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:45.643858 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a3cc1fa2-71bf-4cf2-841c-c37ea9bfcd83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16a4f447415341e49549de51a12645b0'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:45.664697 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56917843-c55b-432b-9d1a-a847ba13aa7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:45.665408 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56917843-c55b-432b-9d1a-a847ba13aa7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:45.666458 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56917843-c55b-432b-9d1a-a847ba13aa7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:45.667083 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56917843-c55b-432b-9d1a-a847ba13aa7e admin admin] PATH_INFO: `/v3/projects/af52d2221ddc4cb8aded94f078049f32` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:45.668660 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56917843-c55b-432b-9d1a-a847ba13aa7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=af52d2221ddc4cb8aded94f078049f32)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:45.670737 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56917843-c55b-432b-9d1a-a847ba13aa7e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:45.685276 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1239/2457] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:23:45 2022] DELETE /identity/v3/users/16a4f447415341e49549de51a12645b0 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:45.689943 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3900bdc-d75d-4d33-a369-ec99a568b24e None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:45.690397 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3900bdc-d75d-4d33-a369-ec99a568b24e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:45.691195 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3900bdc-d75d-4d33-a369-ec99a568b24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:45.716033 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-d3900bdc-d75d-4d33-a369-ec99a568b24e None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:45.717342 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d3900bdc-d75d-4d33-a369-ec99a568b24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:45.734957 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-56917843-c55b-432b-9d1a-a847ba13aa7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af52d2221ddc4cb8aded94f078049f32', 'request_id': 'req-56917843-c55b-432b-9d1a-a847ba13aa7e'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:45.743585 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-56917843-c55b-432b-9d1a-a847ba13aa7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af52d2221ddc4cb8aded94f078049f32', 'request_id': 'req-56917843-c55b-432b-9d1a-a847ba13aa7e'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:45.750999 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-56917843-c55b-432b-9d1a-a847ba13aa7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af52d2221ddc4cb8aded94f078049f32', 'request_id': 'req-56917843-c55b-432b-9d1a-a847ba13aa7e'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:45.770595 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-56917843-c55b-432b-9d1a-a847ba13aa7e admin admin] The token cache is being invalidate because project af52d2221ddc4cb8aded94f078049f32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:23:45.770881 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-56917843-c55b-432b-9d1a-a847ba13aa7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:45.771814 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-56917843-c55b-432b-9d1a-a847ba13aa7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:45.774418 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1219/2458] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:23:45 2022] DELETE /identity/v3/projects/af52d2221ddc4cb8aded94f078049f32 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:45.778468 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56917843-c55b-432b-9d1a-a847ba13aa7e admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:45.780492 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-56917843-c55b-432b-9d1a-a847ba13aa7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:45.799928 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1240/2459] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:23:45.810968 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3900bdc-d75d-4d33-a369-ec99a568b24e None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:45.812526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d3900bdc-d75d-4d33-a369-ec99a568b24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:45.820603 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:45.875765 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56917843-c55b-432b-9d1a-a847ba13aa7e admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:45.884124 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4caf4dc-cd09-4288-98fd-061cf0a7bd43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:45.884124 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4caf4dc-cd09-4288-98fd-061cf0a7bd43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:45.884124 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4caf4dc-cd09-4288-98fd-061cf0a7bd43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:45.884648 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4caf4dc-cd09-4288-98fd-061cf0a7bd43 admin admin] PATH_INFO: `/v3/users/3553d5b59d7b4e1ca84e22cd995e5624` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:45.886746 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4caf4dc-cd09-4288-98fd-061cf0a7bd43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3553d5b59d7b4e1ca84e22cd995e5624)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:45.890229 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4caf4dc-cd09-4288-98fd-061cf0a7bd43 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:45.895532 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3900bdc-d75d-4d33-a369-ec99a568b24e None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:45.899809 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6b6d59-83c0-4d83-a85e-556f12f73cdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:45.900930 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b6d59-83c0-4d83-a85e-556f12f73cdf service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:45.901284 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b6d59-83c0-4d83-a85e-556f12f73cdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:45.901651 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b6d59-83c0-4d83-a85e-556f12f73cdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:45.912074 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6b6d59-83c0-4d83-a85e-556f12f73cdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:45.913875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6b6d59-83c0-4d83-a85e-556f12f73cdf service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:45.929801 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1241/2460] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 04:23:45.949116 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a4caf4dc-cd09-4288-98fd-061cf0a7bd43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3553d5b59d7b4e1ca84e22cd995e5624'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:45.950042 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a4caf4dc-cd09-4288-98fd-061cf0a7bd43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3553d5b59d7b4e1ca84e22cd995e5624'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:45.960423 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a4caf4dc-cd09-4288-98fd-061cf0a7bd43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3553d5b59d7b4e1ca84e22cd995e5624'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:45.987211 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a4caf4dc-cd09-4288-98fd-061cf0a7bd43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3553d5b59d7b4e1ca84e22cd995e5624'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:45.999137 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-a4caf4dc-cd09-4288-98fd-061cf0a7bd43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3553d5b59d7b4e1ca84e22cd995e5624'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:46.012176 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1220/2461] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:23:45 2022] DELETE /identity/v3/users/3553d5b59d7b4e1ca84e22cd995e5624 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:46.480414 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6b6d59-83c0-4d83-a85e-556f12f73cdf service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:46.502540 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6b6d59-83c0-4d83-a85e-556f12f73cdf service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:46.586414 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:46.623385 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4caf4dc-cd09-4288-98fd-061cf0a7bd43 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:46.625051 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-a4caf4dc-cd09-4288-98fd-061cf0a7bd43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:46.630009 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c7c50-d677-49c5-9e3a-6dba1b33191a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:46.630944 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c7c50-d677-49c5-9e3a-6dba1b33191a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:46.631383 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c7c50-d677-49c5-9e3a-6dba1b33191a admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:46.631904 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c7c50-d677-49c5-9e3a-6dba1b33191a admin admin] PATH_INFO: `/v3/projects/e433f0344c18453288e68a1420d09586` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:46.633715 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c7c50-d677-49c5-9e3a-6dba1b33191a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e433f0344c18453288e68a1420d09586)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:46.636404 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c7c50-d677-49c5-9e3a-6dba1b33191a admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:46.662324 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4caf4dc-cd09-4288-98fd-061cf0a7bd43 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:46.670449 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a8290-943a-427a-a983-4e7c46bfc989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:46.679243 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a8290-943a-427a-a983-4e7c46bfc989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:46.680039 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a8290-943a-427a-a983-4e7c46bfc989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:46.680629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a8290-943a-427a-a983-4e7c46bfc989 admin admin] PATH_INFO: `/v3/projects/7667031f3bd942c795cf3238ed9fbcf2` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:46.684008 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6a8290-943a-427a-a983-4e7c46bfc989 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7667031f3bd942c795cf3238ed9fbcf2)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:46.686317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6a8290-943a-427a-a983-4e7c46bfc989 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:46.771178 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-306c7c50-d677-49c5-9e3a-6dba1b33191a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e433f0344c18453288e68a1420d09586', 'request_id': 'req-306c7c50-d677-49c5-9e3a-6dba1b33191a'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:46.784380 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-306c7c50-d677-49c5-9e3a-6dba1b33191a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e433f0344c18453288e68a1420d09586', 'request_id': 'req-306c7c50-d677-49c5-9e3a-6dba1b33191a'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:46.794255 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-5a6a8290-943a-427a-a983-4e7c46bfc989 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7667031f3bd942c795cf3238ed9fbcf2', 'request_id': 'req-5a6a8290-943a-427a-a983-4e7c46bfc989'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:46.795190 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-306c7c50-d677-49c5-9e3a-6dba1b33191a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e433f0344c18453288e68a1420d09586', 'request_id': 'req-306c7c50-d677-49c5-9e3a-6dba1b33191a'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:46.801391 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-5a6a8290-943a-427a-a983-4e7c46bfc989 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7667031f3bd942c795cf3238ed9fbcf2', 'request_id': 'req-5a6a8290-943a-427a-a983-4e7c46bfc989'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:46.808457 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-5a6a8290-943a-427a-a983-4e7c46bfc989 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7667031f3bd942c795cf3238ed9fbcf2', 'request_id': 'req-5a6a8290-943a-427a-a983-4e7c46bfc989'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:46.813946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-306c7c50-d677-49c5-9e3a-6dba1b33191a admin admin] The token cache is being invalidate because project e433f0344c18453288e68a1420d09586 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:23:46.814882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-306c7c50-d677-49c5-9e3a-6dba1b33191a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:46.815746 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-306c7c50-d677-49c5-9e3a-6dba1b33191a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:46.817602 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1242/2462] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:23:46 2022] DELETE /identity/v3/projects/e433f0344c18453288e68a1420d09586 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:46.822857 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c7c50-d677-49c5-9e3a-6dba1b33191a admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:46.824048 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-5a6a8290-943a-427a-a983-4e7c46bfc989 admin admin] The token cache is being invalidate because project 7667031f3bd942c795cf3238ed9fbcf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:23:46.824536 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-5a6a8290-943a-427a-a983-4e7c46bfc989 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:46.825016 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-306c7c50-d677-49c5-9e3a-6dba1b33191a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:46.825385 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-5a6a8290-943a-427a-a983-4e7c46bfc989 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:46.827466 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1221/2463] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:23:46 2022] DELETE /identity/v3/projects/7667031f3bd942c795cf3238ed9fbcf2 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:46.926762 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c7c50-d677-49c5-9e3a-6dba1b33191a admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:47.029682 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6050924a-712c-41ce-98bd-4861d92a7c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:47.031154 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050924a-712c-41ce-98bd-4861d92a7c89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:47.031154 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050924a-712c-41ce-98bd-4861d92a7c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:47.031154 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050924a-712c-41ce-98bd-4861d92a7c89 admin admin] PATH_INFO: `/v3/users/bdde139da26f46aab25091f696594513` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:47.032750 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6050924a-712c-41ce-98bd-4861d92a7c89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdde139da26f46aab25091f696594513)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:47.034605 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6050924a-712c-41ce-98bd-4861d92a7c89 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:47.055455 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:47.092163 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-6050924a-712c-41ce-98bd-4861d92a7c89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdde139da26f46aab25091f696594513'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:47.092835 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-6050924a-712c-41ce-98bd-4861d92a7c89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdde139da26f46aab25091f696594513'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:47.093472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-6050924a-712c-41ce-98bd-4861d92a7c89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdde139da26f46aab25091f696594513'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:47.107202 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-6050924a-712c-41ce-98bd-4861d92a7c89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdde139da26f46aab25091f696594513'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:47.114076 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-6050924a-712c-41ce-98bd-4861d92a7c89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdde139da26f46aab25091f696594513'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:47.127342 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1243/2464] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:23:46 2022] DELETE /identity/v3/users/bdde139da26f46aab25091f696594513 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:47.423687 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a8290-943a-427a-a983-4e7c46bfc989 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:47.425429 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-5a6a8290-943a-427a-a983-4e7c46bfc989 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:47.439196 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a8290-943a-427a-a983-4e7c46bfc989 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:47.472839 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:47.508596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da356d1-2e0d-4bcc-88c6-691253b8c86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:47.509248 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da356d1-2e0d-4bcc-88c6-691253b8c86d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:47.509555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da356d1-2e0d-4bcc-88c6-691253b8c86d admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:47.510414 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da356d1-2e0d-4bcc-88c6-691253b8c86d admin admin] PATH_INFO: `/v3/projects/3a36ecb9ca2b467da57bb3a3c6179709` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:47.511587 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da356d1-2e0d-4bcc-88c6-691253b8c86d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a36ecb9ca2b467da57bb3a3c6179709)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:47.513507 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da356d1-2e0d-4bcc-88c6-691253b8c86d admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:47.561149 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2da356d1-2e0d-4bcc-88c6-691253b8c86d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a36ecb9ca2b467da57bb3a3c6179709', 'request_id': 'req-2da356d1-2e0d-4bcc-88c6-691253b8c86d'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:47.567224 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2da356d1-2e0d-4bcc-88c6-691253b8c86d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a36ecb9ca2b467da57bb3a3c6179709', 'request_id': 'req-2da356d1-2e0d-4bcc-88c6-691253b8c86d'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:47.573588 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2da356d1-2e0d-4bcc-88c6-691253b8c86d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a36ecb9ca2b467da57bb3a3c6179709', 'request_id': 'req-2da356d1-2e0d-4bcc-88c6-691253b8c86d'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:47.585380 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2da356d1-2e0d-4bcc-88c6-691253b8c86d admin admin] The token cache is being invalidate because project 3a36ecb9ca2b467da57bb3a3c6179709 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:23:47.585669 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2da356d1-2e0d-4bcc-88c6-691253b8c86d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:47.586323 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2da356d1-2e0d-4bcc-88c6-691253b8c86d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:47.588502 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1222/2465] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:23:47 2022] DELETE /identity/v3/projects/3a36ecb9ca2b467da57bb3a3c6179709 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:47.593459 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b68acb-1f09-4881-a227-615c670c1dbd None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:47.593745 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b68acb-1f09-4881-a227-615c670c1dbd None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:47.594045 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b68acb-1f09-4881-a227-615c670c1dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:47.613558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-35b68acb-1f09-4881-a227-615c670c1dbd None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:47.614806 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-35b68acb-1f09-4881-a227-615c670c1dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:47.679711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1244/2466] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:47.688855 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0599c-2887-49d4-b13f-2da299856f2b None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:47.689320 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0599c-2887-49d4-b13f-2da299856f2b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:47.689416 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0599c-2887-49d4-b13f-2da299856f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:47.710429 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-c8c0599c-2887-49d4-b13f-2da299856f2b None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:47.711568 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c8c0599c-2887-49d4-b13f-2da299856f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:47.720500 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1223/2467] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:47.724721 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b68acb-1f09-4881-a227-615c670c1dbd None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:47.735623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b68acb-1f09-4881-a227-615c670c1dbd None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:47.740690 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0640fe36-dd4e-415e-a475-37c5404294dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:47.741774 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640fe36-dd4e-415e-a475-37c5404294dc admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:47.741774 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640fe36-dd4e-415e-a475-37c5404294dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:47.742117 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640fe36-dd4e-415e-a475-37c5404294dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:47.742637 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0640fe36-dd4e-415e-a475-37c5404294dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:23:47.742952 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0640fe36-dd4e-415e-a475-37c5404294dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:47.744641 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0640fe36-dd4e-415e-a475-37c5404294dc admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:47.760910 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1245/2468] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:23:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:23:47.762136 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c0599c-2887-49d4-b13f-2da299856f2b None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:47.771572 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c0599c-2887-49d4-b13f-2da299856f2b None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:47.777177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50143e9-cf82-4324-b749-0d876f08e97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:47.777669 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50143e9-cf82-4324-b749-0d876f08e97a admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:47.777950 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50143e9-cf82-4324-b749-0d876f08e97a admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:47.778263 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50143e9-cf82-4324-b749-0d876f08e97a admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:47.778942 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50143e9-cf82-4324-b749-0d876f08e97a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:47.781931 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50143e9-cf82-4324-b749-0d876f08e97a admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:47.810653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1224/2469] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:23:47 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:47.813858 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0640fe36-dd4e-415e-a475-37c5404294dc admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:47.825245 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0640fe36-dd4e-415e-a475-37c5404294dc admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:47.847707 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50143e9-cf82-4324-b749-0d876f08e97a admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:47.849383 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f50143e9-cf82-4324-b749-0d876f08e97a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:47.893302 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece56364-224a-4e1b-be81-77cdf8faf94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:47.893925 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece56364-224a-4e1b-be81-77cdf8faf94e admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:47.894241 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece56364-224a-4e1b-be81-77cdf8faf94e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:47.894525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece56364-224a-4e1b-be81-77cdf8faf94e admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:47.895206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece56364-224a-4e1b-be81-77cdf8faf94e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:47.897182 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece56364-224a-4e1b-be81-77cdf8faf94e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:47.929139 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50143e9-cf82-4324-b749-0d876f08e97a admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:47.929839 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1246/2470] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:23:47 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:23:47.930960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece56364-224a-4e1b-be81-77cdf8faf94e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:47.934615 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec21af2f-0734-4aad-9bf8-47dbed2ed96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:47.935269 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec21af2f-0734-4aad-9bf8-47dbed2ed96c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:47.935548 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec21af2f-0734-4aad-9bf8-47dbed2ed96c admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:47.935808 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec21af2f-0734-4aad-9bf8-47dbed2ed96c admin admin] PATH_INFO: `/v3/users/00668826ceba4fcb8cb245a0cd271026` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:47.937017 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec21af2f-0734-4aad-9bf8-47dbed2ed96c admin admin] RBAC: Authorizing `identity:delete_user(user_id=00668826ceba4fcb8cb245a0cd271026)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:47.938176 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:47.939133 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec21af2f-0734-4aad-9bf8-47dbed2ed96c admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:47.942326 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece56364-224a-4e1b-be81-77cdf8faf94e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:47.950675 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083998f6-40c6-4a75-bdfc-18dc2e5c7943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:47.950892 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083998f6-40c6-4a75-bdfc-18dc2e5c7943 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:47.951135 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083998f6-40c6-4a75-bdfc-18dc2e5c7943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:47.951413 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083998f6-40c6-4a75-bdfc-18dc2e5c7943 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:47.952451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083998f6-40c6-4a75-bdfc-18dc2e5c7943 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:47.954558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083998f6-40c6-4a75-bdfc-18dc2e5c7943 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:47.966151 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1247/2471] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:47.970766 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083998f6-40c6-4a75-bdfc-18dc2e5c7943 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:47.977374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:47.990179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083998f6-40c6-4a75-bdfc-18dc2e5c7943 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:47.996680 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613f0f4-a5a6-4611-9954-2ce324a95857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:47.997613 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613f0f4-a5a6-4611-9954-2ce324a95857 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:47.998035 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613f0f4-a5a6-4611-9954-2ce324a95857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:47.998338 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613f0f4-a5a6-4611-9954-2ce324a95857 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:47.998967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ec21af2f-0734-4aad-9bf8-47dbed2ed96c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00668826ceba4fcb8cb245a0cd271026'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:47.999105 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1613f0f4-a5a6-4611-9954-2ce324a95857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:47.999679 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ec21af2f-0734-4aad-9bf8-47dbed2ed96c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00668826ceba4fcb8cb245a0cd271026'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:48.000619 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1613f0f4-a5a6-4611-9954-2ce324a95857 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:48.008302 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ec21af2f-0734-4aad-9bf8-47dbed2ed96c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00668826ceba4fcb8cb245a0cd271026'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:48.014012 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1248/2472] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:47 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 04:23:48.017427 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613f0f4-a5a6-4611-9954-2ce324a95857 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:48.026578 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ec21af2f-0734-4aad-9bf8-47dbed2ed96c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00668826ceba4fcb8cb245a0cd271026'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:48.029927 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613f0f4-a5a6-4611-9954-2ce324a95857 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:48.034166 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ec21af2f-0734-4aad-9bf8-47dbed2ed96c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00668826ceba4fcb8cb245a0cd271026'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:48.035333 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38dabe-aceb-4708-a5c2-0ad0a052a0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:48.036120 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38dabe-aceb-4708-a5c2-0ad0a052a0ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:48.036521 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38dabe-aceb-4708-a5c2-0ad0a052a0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:48.036803 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38dabe-aceb-4708-a5c2-0ad0a052a0ae admin admin] PATH_INFO: `/v3/projects/2bdf3b15074644df9b51e1fa0b78b28a/users/8f6c61f952c1472eaeb8764fcbd39801/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:48.041553 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1225/2473] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:23:47 2022] DELETE /identity/v3/users/00668826ceba4fcb8cb245a0cd271026 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:23:48.046308 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310e601-ce5e-4c88-9b24-ef41d39d5a36 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:48.046601 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310e601-ce5e-4c88-9b24-ef41d39d5a36 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:48.046881 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310e601-ce5e-4c88-9b24-ef41d39d5a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:48.054536 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38dabe-aceb-4708-a5c2-0ad0a052a0ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bdf3b15074644df9b51e1fa0b78b28a, user_id=8f6c61f952c1472eaeb8764fcbd39801, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:48.058550 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38dabe-aceb-4708-a5c2-0ad0a052a0ae admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:48.067092 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-0310e601-ce5e-4c88-9b24-ef41d39d5a36 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:48.068304 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0310e601-ce5e-4c88-9b24-ef41d39d5a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:48.074861 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1249/2474] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:23:48 2022] PUT /identity/v3/projects/2bdf3b15074644df9b51e1fa0b78b28a/users/8f6c61f952c1472eaeb8764fcbd39801/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:23:48.085020 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330facc-3824-4d45-8b67-4cf4eefa1e5a None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:48.085341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330facc-3824-4d45-8b67-4cf4eefa1e5a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:48.085630 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330facc-3824-4d45-8b67-4cf4eefa1e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:48.136757 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-8330facc-3824-4d45-8b67-4cf4eefa1e5a None None] MFA Rules not processed for user `8f6c61f952c1472eaeb8764fcbd39801`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:48.138241 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8330facc-3824-4d45-8b67-4cf4eefa1e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:48.148841 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1226/2475] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:48.158963 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310e601-ce5e-4c88-9b24-ef41d39d5a36 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:48.160411 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0310e601-ce5e-4c88-9b24-ef41d39d5a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:48.238014 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1250/2476] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:48.245078 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310e601-ce5e-4c88-9b24-ef41d39d5a36 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:48.247183 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330facc-3824-4d45-8b67-4cf4eefa1e5a None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:48.248558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8330facc-3824-4d45-8b67-4cf4eefa1e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:48.248928 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c07b2-2d78-4007-91cd-595138edb229 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:48.250233 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c07b2-2d78-4007-91cd-595138edb229 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:48.250517 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c07b2-2d78-4007-91cd-595138edb229 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:48.250963 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c07b2-2d78-4007-91cd-595138edb229 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:48.262082 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c07b2-2d78-4007-91cd-595138edb229 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:48.263629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c07b2-2d78-4007-91cd-595138edb229 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:48.280229 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1227/2477] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 04:23:48.334309 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330facc-3824-4d45-8b67-4cf4eefa1e5a None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:48.338536 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d4f791-ee32-49a5-a61a-4754fd1d75d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:48.339193 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4f791-ee32-49a5-a61a-4754fd1d75d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:48.339476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4f791-ee32-49a5-a61a-4754fd1d75d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:48.339752 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4f791-ee32-49a5-a61a-4754fd1d75d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:48.348660 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d4f791-ee32-49a5-a61a-4754fd1d75d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:48.350149 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d4f791-ee32-49a5-a61a-4754fd1d75d9 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:48.360267 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1251/2478] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 04:23:48.571333 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c07b2-2d78-4007-91cd-595138edb229 service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:48.572872 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ff3c07b2-2d78-4007-91cd-595138edb229 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:48.596347 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c07b2-2d78-4007-91cd-595138edb229 service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:48.604243 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d454f39-4af7-409a-a1c5-8a664b7cffd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:48.605377 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d454f39-4af7-409a-a1c5-8a664b7cffd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:48.605675 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d454f39-4af7-409a-a1c5-8a664b7cffd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:48.606342 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d454f39-4af7-409a-a1c5-8a664b7cffd1 admin admin] PATH_INFO: `/v3/projects/f60af79582d4481babcb9aefb1afb657` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:48.607564 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d454f39-4af7-409a-a1c5-8a664b7cffd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f60af79582d4481babcb9aefb1afb657)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:48.609489 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d454f39-4af7-409a-a1c5-8a664b7cffd1 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:48.678608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:48.687126 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0d454f39-4af7-409a-a1c5-8a664b7cffd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f60af79582d4481babcb9aefb1afb657', 'request_id': 'req-0d454f39-4af7-409a-a1c5-8a664b7cffd1'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:48.698214 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0d454f39-4af7-409a-a1c5-8a664b7cffd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f60af79582d4481babcb9aefb1afb657', 'request_id': 'req-0d454f39-4af7-409a-a1c5-8a664b7cffd1'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:48.701777 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0d454f39-4af7-409a-a1c5-8a664b7cffd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f60af79582d4481babcb9aefb1afb657', 'request_id': 'req-0d454f39-4af7-409a-a1c5-8a664b7cffd1'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:48.716951 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0d454f39-4af7-409a-a1c5-8a664b7cffd1 admin admin] The token cache is being invalidate because project f60af79582d4481babcb9aefb1afb657 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:23:48.717277 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0d454f39-4af7-409a-a1c5-8a664b7cffd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:48.717919 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-0d454f39-4af7-409a-a1c5-8a664b7cffd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:48.719595 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1228/2479] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:23:48 2022] DELETE /identity/v3/projects/f60af79582d4481babcb9aefb1afb657 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:48.723636 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d4f791-ee32-49a5-a61a-4754fd1d75d9 service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:48.735212 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d4f791-ee32-49a5-a61a-4754fd1d75d9 service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:48.796205 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:48.806647 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f0379f-7034-4603-8153-145025653121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:48.807729 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f0379f-7034-4603-8153-145025653121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:48.808239 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f0379f-7034-4603-8153-145025653121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:48.808523 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f0379f-7034-4603-8153-145025653121 admin admin] PATH_INFO: `/v3/users/affcb20277b443ba98ddb4744f59c0e7` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:48.809751 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f0379f-7034-4603-8153-145025653121 admin admin] RBAC: Authorizing `identity:delete_user(user_id=affcb20277b443ba98ddb4744f59c0e7)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:48.811760 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f0379f-7034-4603-8153-145025653121 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:48.858617 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d8f0379f-7034-4603-8153-145025653121 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'affcb20277b443ba98ddb4744f59c0e7'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:48.859303 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d8f0379f-7034-4603-8153-145025653121 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'affcb20277b443ba98ddb4744f59c0e7'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:48.860367 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d8f0379f-7034-4603-8153-145025653121 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'affcb20277b443ba98ddb4744f59c0e7'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:48.872950 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d8f0379f-7034-4603-8153-145025653121 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'affcb20277b443ba98ddb4744f59c0e7'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:48.880296 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d8f0379f-7034-4603-8153-145025653121 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'affcb20277b443ba98ddb4744f59c0e7'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:48.894313 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1252/2480] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:23:48 2022] DELETE /identity/v3/users/affcb20277b443ba98ddb4744f59c0e7 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:49.171234 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d454f39-4af7-409a-a1c5-8a664b7cffd1 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:49.172676 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0d454f39-4af7-409a-a1c5-8a664b7cffd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:49.188777 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d454f39-4af7-409a-a1c5-8a664b7cffd1 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:49.248225 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6b40e-a492-413f-8c78-db75fa3005a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:49.248882 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6b40e-a492-413f-8c78-db75fa3005a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:49.249223 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6b40e-a492-413f-8c78-db75fa3005a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:49.249446 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6b40e-a492-413f-8c78-db75fa3005a3 admin admin] PATH_INFO: `/v3/projects/7f919dc8ddbe444684d3aa849bf045aa` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:49.250614 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6b40e-a492-413f-8c78-db75fa3005a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f919dc8ddbe444684d3aa849bf045aa)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:49.252557 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6b40e-a492-413f-8c78-db75fa3005a3 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:49.301656 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-06e6b40e-a492-413f-8c78-db75fa3005a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f919dc8ddbe444684d3aa849bf045aa', 'request_id': 'req-06e6b40e-a492-413f-8c78-db75fa3005a3'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:49.308205 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-06e6b40e-a492-413f-8c78-db75fa3005a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f919dc8ddbe444684d3aa849bf045aa', 'request_id': 'req-06e6b40e-a492-413f-8c78-db75fa3005a3'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:49.313721 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-06e6b40e-a492-413f-8c78-db75fa3005a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f919dc8ddbe444684d3aa849bf045aa', 'request_id': 'req-06e6b40e-a492-413f-8c78-db75fa3005a3'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:49.327684 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-06e6b40e-a492-413f-8c78-db75fa3005a3 admin admin] The token cache is being invalidate because project 7f919dc8ddbe444684d3aa849bf045aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:23:49.328038 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-06e6b40e-a492-413f-8c78-db75fa3005a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:49.328660 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-06e6b40e-a492-413f-8c78-db75fa3005a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:49.330218 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1229/2481] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:23:49 2022] DELETE /identity/v3/projects/7f919dc8ddbe444684d3aa849bf045aa => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:49.334403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f0379f-7034-4603-8153-145025653121 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:49.335840 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d8f0379f-7034-4603-8153-145025653121 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:49.349885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f0379f-7034-4603-8153-145025653121 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:49.408289 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9371038-4325-44e9-9446-84d6f080f9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:49.408939 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9371038-4325-44e9-9446-84d6f080f9c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:49.409218 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9371038-4325-44e9-9446-84d6f080f9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:49.409491 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9371038-4325-44e9-9446-84d6f080f9c5 admin admin] PATH_INFO: `/v3/users/b3d7f445420249f8b2967cab16327f96` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:49.410677 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9371038-4325-44e9-9446-84d6f080f9c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3d7f445420249f8b2967cab16327f96)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:49.412583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9371038-4325-44e9-9446-84d6f080f9c5 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:49.466253 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c9371038-4325-44e9-9446-84d6f080f9c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3d7f445420249f8b2967cab16327f96'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:49.467055 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c9371038-4325-44e9-9446-84d6f080f9c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3d7f445420249f8b2967cab16327f96'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:49.467744 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c9371038-4325-44e9-9446-84d6f080f9c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3d7f445420249f8b2967cab16327f96'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:49.485219 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c9371038-4325-44e9-9446-84d6f080f9c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3d7f445420249f8b2967cab16327f96'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:49.491042 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:49.493412 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c9371038-4325-44e9-9446-84d6f080f9c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3d7f445420249f8b2967cab16327f96'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:49.508336 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1253/2482] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:23:49 2022] DELETE /identity/v3/users/b3d7f445420249f8b2967cab16327f96 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:49.792285 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6b40e-a492-413f-8c78-db75fa3005a3 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:49.793910 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-06e6b40e-a492-413f-8c78-db75fa3005a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:49.807840 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6b40e-a492-413f-8c78-db75fa3005a3 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:49.872754 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005f21ca-71be-490e-ab34-09eacf9d704f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:49.873410 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f21ca-71be-490e-ab34-09eacf9d704f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:49.873695 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f21ca-71be-490e-ab34-09eacf9d704f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:49.873995 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f21ca-71be-490e-ab34-09eacf9d704f admin admin] PATH_INFO: `/v3/projects/3a4cde7208c04b68af7984735be38fb2` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:49.875207 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005f21ca-71be-490e-ab34-09eacf9d704f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a4cde7208c04b68af7984735be38fb2)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:49.877100 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005f21ca-71be-490e-ab34-09eacf9d704f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:49.883405 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:49.930492 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-005f21ca-71be-490e-ab34-09eacf9d704f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a4cde7208c04b68af7984735be38fb2', 'request_id': 'req-005f21ca-71be-490e-ab34-09eacf9d704f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:49.938583 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-005f21ca-71be-490e-ab34-09eacf9d704f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a4cde7208c04b68af7984735be38fb2', 'request_id': 'req-005f21ca-71be-490e-ab34-09eacf9d704f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:49.944759 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-005f21ca-71be-490e-ab34-09eacf9d704f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a4cde7208c04b68af7984735be38fb2', 'request_id': 'req-005f21ca-71be-490e-ab34-09eacf9d704f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:49.960470 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-005f21ca-71be-490e-ab34-09eacf9d704f admin admin] The token cache is being invalidate because project 3a4cde7208c04b68af7984735be38fb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:23:49.960838 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-005f21ca-71be-490e-ab34-09eacf9d704f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:49.961636 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-005f21ca-71be-490e-ab34-09eacf9d704f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:49.963602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1230/2483] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:23:49 2022] DELETE /identity/v3/projects/3a4cde7208c04b68af7984735be38fb2 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:49.971790 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660eeed4-11c7-45fc-84bb-f034e3abd0e2 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:49.972478 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660eeed4-11c7-45fc-84bb-f034e3abd0e2 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:49.972478 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660eeed4-11c7-45fc-84bb-f034e3abd0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:49.997773 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-660eeed4-11c7-45fc-84bb-f034e3abd0e2 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:49.999012 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-660eeed4-11c7-45fc-84bb-f034e3abd0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:50.058700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1254/2484] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:23:50.067922 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c4c05-9ca5-4e75-b725-a889d1c0eeee None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:50.068402 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c4c05-9ca5-4e75-b725-a889d1c0eeee None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:50.068766 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c4c05-9ca5-4e75-b725-a889d1c0eeee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:50.090178 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-672c4c05-9ca5-4e75-b725-a889d1c0eeee None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:50.091401 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-672c4c05-9ca5-4e75-b725-a889d1c0eeee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:50.099583 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1231/2485] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:50.104002 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660eeed4-11c7-45fc-84bb-f034e3abd0e2 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:50.114058 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660eeed4-11c7-45fc-84bb-f034e3abd0e2 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:50.119303 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439755ca-63ed-4cea-a56a-31faedd1a98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:50.120031 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439755ca-63ed-4cea-a56a-31faedd1a98a admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:50.120497 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439755ca-63ed-4cea-a56a-31faedd1a98a admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:50.120994 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439755ca-63ed-4cea-a56a-31faedd1a98a admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:50.121752 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439755ca-63ed-4cea-a56a-31faedd1a98a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:23:50.122098 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439755ca-63ed-4cea-a56a-31faedd1a98a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:50.123719 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439755ca-63ed-4cea-a56a-31faedd1a98a admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:50.142503 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1255/2486] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:23:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:23:50.147183 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c4c05-9ca5-4e75-b725-a889d1c0eeee None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:50.157703 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c4c05-9ca5-4e75-b725-a889d1c0eeee None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:50.162985 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d528ea1-40de-48af-ba74-665070b45fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:50.163604 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d528ea1-40de-48af-ba74-665070b45fc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:50.163890 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d528ea1-40de-48af-ba74-665070b45fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:50.164167 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d528ea1-40de-48af-ba74-665070b45fc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:50.164851 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d528ea1-40de-48af-ba74-665070b45fc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:50.167634 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d528ea1-40de-48af-ba74-665070b45fc6 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:50.197288 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1232/2487] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:23:50 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:50.201725 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439755ca-63ed-4cea-a56a-31faedd1a98a admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:50.217261 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439755ca-63ed-4cea-a56a-31faedd1a98a admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:50.290250 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73761c7d-9d9d-44de-9771-40b336ca8512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:50.291728 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73761c7d-9d9d-44de-9771-40b336ca8512 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:50.292251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73761c7d-9d9d-44de-9771-40b336ca8512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:50.292799 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73761c7d-9d9d-44de-9771-40b336ca8512 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:50.293875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73761c7d-9d9d-44de-9771-40b336ca8512 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:50.296572 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73761c7d-9d9d-44de-9771-40b336ca8512 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:50.327334 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1256/2488] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:23:50 2022] POST /identity/v3/users => generated 287 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:50.328600 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d528ea1-40de-48af-ba74-665070b45fc6 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:50.340944 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d528ea1-40de-48af-ba74-665070b45fc6 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:50.346045 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa31ce2-a227-4cbd-a414-561eb9920f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:50.347099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa31ce2-a227-4cbd-a414-561eb9920f3e admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:50.347390 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa31ce2-a227-4cbd-a414-561eb9920f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:50.347672 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa31ce2-a227-4cbd-a414-561eb9920f3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:50.348844 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa31ce2-a227-4cbd-a414-561eb9920f3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:50.351256 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa31ce2-a227-4cbd-a414-561eb9920f3e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:50.366916 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1233/2489] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:50 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 04:23:50.370929 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73761c7d-9d9d-44de-9771-40b336ca8512 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:50.381348 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73761c7d-9d9d-44de-9771-40b336ca8512 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:50.386809 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4123ad-cece-4ba5-9315-db510f0d0510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:50.387446 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4123ad-cece-4ba5-9315-db510f0d0510 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:50.387729 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4123ad-cece-4ba5-9315-db510f0d0510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:50.388002 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4123ad-cece-4ba5-9315-db510f0d0510 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:50.388724 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4123ad-cece-4ba5-9315-db510f0d0510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:50.390395 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4123ad-cece-4ba5-9315-db510f0d0510 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:50.403220 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1257/2490] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:50.407597 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa31ce2-a227-4cbd-a414-561eb9920f3e admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:50.419647 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa31ce2-a227-4cbd-a414-561eb9920f3e admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:50.425450 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7759547a-a759-441a-9869-b6338f4d8cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:50.426193 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7759547a-a759-441a-9869-b6338f4d8cbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:50.426484 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7759547a-a759-441a-9869-b6338f4d8cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:50.426764 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7759547a-a759-441a-9869-b6338f4d8cbb admin admin] PATH_INFO: `/v3/projects/992437dfa6ef4c678ea5d4f92155cf17/users/5a7f399240ee4003b0e72fd34d96f47b/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:50.443586 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7759547a-a759-441a-9869-b6338f4d8cbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=992437dfa6ef4c678ea5d4f92155cf17, user_id=5a7f399240ee4003b0e72fd34d96f47b, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:50.448332 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7759547a-a759-441a-9869-b6338f4d8cbb admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:50.464058 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1234/2491] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:23:50 2022] PUT /identity/v3/projects/992437dfa6ef4c678ea5d4f92155cf17/users/5a7f399240ee4003b0e72fd34d96f47b/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:23:50.469025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c06299-d61e-4a1e-8c70-908db00c7b15 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:50.469318 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c06299-d61e-4a1e-8c70-908db00c7b15 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:50.469598 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c06299-d61e-4a1e-8c70-908db00c7b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:50.492526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-60c06299-d61e-4a1e-8c70-908db00c7b15 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:50.494294 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-60c06299-d61e-4a1e-8c70-908db00c7b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:50.558009 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:50.576678 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1258/2492] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:50.589780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7759547a-a759-441a-9869-b6338f4d8cbb admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:50.591654 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-7759547a-a759-441a-9869-b6338f4d8cbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:50.664642 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7759547a-a759-441a-9869-b6338f4d8cbb admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:50.671406 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8cc5ce-02b0-47c5-9226-6c95d97501ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:50.672224 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8cc5ce-02b0-47c5-9226-6c95d97501ac service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:50.672516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8cc5ce-02b0-47c5-9226-6c95d97501ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:50.672889 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8cc5ce-02b0-47c5-9226-6c95d97501ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:50.685425 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8cc5ce-02b0-47c5-9226-6c95d97501ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:50.687273 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8cc5ce-02b0-47c5-9226-6c95d97501ac service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:50.700860 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1235/2493] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:23:52.107171 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c06299-d61e-4a1e-8c70-908db00c7b15 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:52.110173 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-60c06299-d61e-4a1e-8c70-908db00c7b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:52.129858 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c06299-d61e-4a1e-8c70-908db00c7b15 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:52.134512 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f45cff-58c7-4ddb-989c-5ed89b9e8894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:52.135253 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f45cff-58c7-4ddb-989c-5ed89b9e8894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:52.135437 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f45cff-58c7-4ddb-989c-5ed89b9e8894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:52.136045 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f45cff-58c7-4ddb-989c-5ed89b9e8894 admin admin] PATH_INFO: `/v3/users/8f6c61f952c1472eaeb8764fcbd39801` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:52.140579 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f45cff-58c7-4ddb-989c-5ed89b9e8894 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f6c61f952c1472eaeb8764fcbd39801)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:52.140816 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f45cff-58c7-4ddb-989c-5ed89b9e8894 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:52.217618 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b5f45cff-58c7-4ddb-989c-5ed89b9e8894 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f6c61f952c1472eaeb8764fcbd39801'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:52.218372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b5f45cff-58c7-4ddb-989c-5ed89b9e8894 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f6c61f952c1472eaeb8764fcbd39801'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:52.219053 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b5f45cff-58c7-4ddb-989c-5ed89b9e8894 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f6c61f952c1472eaeb8764fcbd39801'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:52.232949 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b5f45cff-58c7-4ddb-989c-5ed89b9e8894 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f6c61f952c1472eaeb8764fcbd39801'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:52.241620 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b5f45cff-58c7-4ddb-989c-5ed89b9e8894 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f6c61f952c1472eaeb8764fcbd39801'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:52.260827 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1259/2494] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:23:52 2022] DELETE /identity/v3/users/8f6c61f952c1472eaeb8764fcbd39801 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:52.262549 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc5867-5c27-4727-ac71-6b7730733d4d None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:52.263040 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc5867-5c27-4727-ac71-6b7730733d4d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:52.263125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc5867-5c27-4727-ac71-6b7730733d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:52.302815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-44bc5867-5c27-4727-ac71-6b7730733d4d None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:52.304053 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-44bc5867-5c27-4727-ac71-6b7730733d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:52.366765 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:52.427244 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1236/2495] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:52.446059 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f45cff-58c7-4ddb-989c-5ed89b9e8894 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:52.449691 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b5f45cff-58c7-4ddb-989c-5ed89b9e8894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:52.520666 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f45cff-58c7-4ddb-989c-5ed89b9e8894 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:52.524265 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925dc29-d821-456b-9048-155713de88b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:52.524887 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925dc29-d821-456b-9048-155713de88b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:52.525873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925dc29-d821-456b-9048-155713de88b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:52.526293 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925dc29-d821-456b-9048-155713de88b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:52.535737 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6925dc29-d821-456b-9048-155713de88b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:52.537270 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6925dc29-d821-456b-9048-155713de88b6 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:52.547429 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1260/2496] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:23:53.041939 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc5867-5c27-4727-ac71-6b7730733d4d None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:53.047721 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-44bc5867-5c27-4727-ac71-6b7730733d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:53.063048 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc5867-5c27-4727-ac71-6b7730733d4d None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:53.073585 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a359d-d0e2-4f26-b202-88a19ef70146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:53.074598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a359d-d0e2-4f26-b202-88a19ef70146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:53.075856 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a359d-d0e2-4f26-b202-88a19ef70146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:53.076168 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a359d-d0e2-4f26-b202-88a19ef70146 admin admin] PATH_INFO: `/v3/projects/2bdf3b15074644df9b51e1fa0b78b28a` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:53.077621 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572a359d-d0e2-4f26-b202-88a19ef70146 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bdf3b15074644df9b51e1fa0b78b28a)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:53.079850 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572a359d-d0e2-4f26-b202-88a19ef70146 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:53.145469 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-572a359d-d0e2-4f26-b202-88a19ef70146 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bdf3b15074644df9b51e1fa0b78b28a', 'request_id': 'req-572a359d-d0e2-4f26-b202-88a19ef70146'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:53.153272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-572a359d-d0e2-4f26-b202-88a19ef70146 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bdf3b15074644df9b51e1fa0b78b28a', 'request_id': 'req-572a359d-d0e2-4f26-b202-88a19ef70146'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:53.158997 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-572a359d-d0e2-4f26-b202-88a19ef70146 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bdf3b15074644df9b51e1fa0b78b28a', 'request_id': 'req-572a359d-d0e2-4f26-b202-88a19ef70146'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:53.180654 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-572a359d-d0e2-4f26-b202-88a19ef70146 admin admin] The token cache is being invalidate because project 2bdf3b15074644df9b51e1fa0b78b28a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:23:53.180654 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-572a359d-d0e2-4f26-b202-88a19ef70146 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:53.182738 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-572a359d-d0e2-4f26-b202-88a19ef70146 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:23:53.188508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1237/2497] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:23:53 2022] DELETE /identity/v3/projects/2bdf3b15074644df9b51e1fa0b78b28a => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:53.192232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c194134-0b2b-4eeb-b907-42cdc8231506 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:53.192232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c194134-0b2b-4eeb-b907-42cdc8231506 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:53.192232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c194134-0b2b-4eeb-b907-42cdc8231506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:53.216090 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-2c194134-0b2b-4eeb-b907-42cdc8231506 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:53.217351 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2c194134-0b2b-4eeb-b907-42cdc8231506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:53.293010 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1261/2498] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:53.300927 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c09d415-0e8d-4da1-8512-f0e6ff161e25 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:53.301430 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c09d415-0e8d-4da1-8512-f0e6ff161e25 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:53.301549 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c09d415-0e8d-4da1-8512-f0e6ff161e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:53.329687 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-1c09d415-0e8d-4da1-8512-f0e6ff161e25 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:53.330992 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-1c09d415-0e8d-4da1-8512-f0e6ff161e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:53.342051 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1238/2499] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:53.345316 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c194134-0b2b-4eeb-b907-42cdc8231506 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:53.359712 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c194134-0b2b-4eeb-b907-42cdc8231506 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:53.375265 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa4b53-a000-4a83-b3c4-64fe49bdc644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:53.376056 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa4b53-a000-4a83-b3c4-64fe49bdc644 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:53.376472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa4b53-a000-4a83-b3c4-64fe49bdc644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:53.376665 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa4b53-a000-4a83-b3c4-64fe49bdc644 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:53.377559 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fa4b53-a000-4a83-b3c4-64fe49bdc644 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:23:53.378329 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fa4b53-a000-4a83-b3c4-64fe49bdc644 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:53.379597 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fa4b53-a000-4a83-b3c4-64fe49bdc644 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:53.387973 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:53.398298 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1262/2500] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:23:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:23:53.398896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c09d415-0e8d-4da1-8512-f0e6ff161e25 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:53.409900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c09d415-0e8d-4da1-8512-f0e6ff161e25 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:53.424352 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c1030-ebc0-4ed6-adda-d53c9a5841e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:53.424352 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c1030-ebc0-4ed6-adda-d53c9a5841e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:53.424352 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c1030-ebc0-4ed6-adda-d53c9a5841e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:53.425502 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c1030-ebc0-4ed6-adda-d53c9a5841e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:53.426452 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c1030-ebc0-4ed6-adda-d53c9a5841e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:53.430055 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c1030-ebc0-4ed6-adda-d53c9a5841e2 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:53.478954 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1239/2501] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:23:53 2022] POST /identity/v3/projects => generated 375 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:53.481286 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa4b53-a000-4a83-b3c4-64fe49bdc644 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:53.500469 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa4b53-a000-4a83-b3c4-64fe49bdc644 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:53.593553 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191601f-405f-4095-af25-128826b9bf18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:53.594370 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191601f-405f-4095-af25-128826b9bf18 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:53.594705 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191601f-405f-4095-af25-128826b9bf18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:53.595032 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191601f-405f-4095-af25-128826b9bf18 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:53.595726 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f191601f-405f-4095-af25-128826b9bf18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:53.597756 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f191601f-405f-4095-af25-128826b9bf18 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:53.655571 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1263/2502] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:23:53 2022] POST /identity/v3/users => generated 288 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:53.656411 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c1030-ebc0-4ed6-adda-d53c9a5841e2 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:53.668081 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c1030-ebc0-4ed6-adda-d53c9a5841e2 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:53.673287 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a6d49-01a6-4570-b202-de34a47825b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:53.673908 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a6d49-01a6-4570-b202-de34a47825b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:53.674233 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a6d49-01a6-4570-b202-de34a47825b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:53.674512 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a6d49-01a6-4570-b202-de34a47825b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:53.675240 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7a6d49-01a6-4570-b202-de34a47825b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:53.676868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7a6d49-01a6-4570-b202-de34a47825b8 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:53.707263 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191601f-405f-4095-af25-128826b9bf18 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:53.720554 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1240/2503] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:53.726292 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191601f-405f-4095-af25-128826b9bf18 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:53.732180 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c97cc-bc14-45e6-87e6-d717ce33b971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:53.732942 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c97cc-bc14-45e6-87e6-d717ce33b971 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:53.733301 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c97cc-bc14-45e6-87e6-d717ce33b971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:53.733655 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c97cc-bc14-45e6-87e6-d717ce33b971 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:53.734499 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c97cc-bc14-45e6-87e6-d717ce33b971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:53.736222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c97cc-bc14-45e6-87e6-d717ce33b971 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:53.759869 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a6d49-01a6-4570-b202-de34a47825b8 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:53.766246 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1264/2504] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:53.775708 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a6d49-01a6-4570-b202-de34a47825b8 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:53.782531 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce051b-fcaf-4d49-a349-3e3a549fce28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:53.783826 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce051b-fcaf-4d49-a349-3e3a549fce28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:53.784111 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce051b-fcaf-4d49-a349-3e3a549fce28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:53.784391 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce051b-fcaf-4d49-a349-3e3a549fce28 admin admin] PATH_INFO: `/v3/projects/3825c2630a4f47e0b029bd0c90875108/users/eea5b08eede4409484dc2f4305b5298a/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:53.804660 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ce051b-fcaf-4d49-a349-3e3a549fce28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3825c2630a4f47e0b029bd0c90875108, user_id=eea5b08eede4409484dc2f4305b5298a, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:53.807661 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ce051b-fcaf-4d49-a349-3e3a549fce28 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:53.846491 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1241/2505] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:23:53 2022] PUT /identity/v3/projects/3825c2630a4f47e0b029bd0c90875108/users/eea5b08eede4409484dc2f4305b5298a/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:53.848176 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707d9e5-eed7-4657-87d5-755fe22be616 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:53.848539 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707d9e5-eed7-4657-87d5-755fe22be616 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:53.848828 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707d9e5-eed7-4657-87d5-755fe22be616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:53.871513 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-2707d9e5-eed7-4657-87d5-755fe22be616 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:53.872747 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2707d9e5-eed7-4657-87d5-755fe22be616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:53.948975 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1265/2506] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:53.959778 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce051b-fcaf-4d49-a349-3e3a549fce28 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:53.961842 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-21ce051b-fcaf-4d49-a349-3e3a549fce28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:54.040185 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce051b-fcaf-4d49-a349-3e3a549fce28 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:54.045842 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b184d6f-7eeb-4604-854c-77a17cdf8905 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:54.047448 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b184d6f-7eeb-4604-854c-77a17cdf8905 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:54.047448 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b184d6f-7eeb-4604-854c-77a17cdf8905 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:54.047448 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b184d6f-7eeb-4604-854c-77a17cdf8905 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:54.058278 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:54.060492 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b184d6f-7eeb-4604-854c-77a17cdf8905 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:54.062049 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b184d6f-7eeb-4604-854c-77a17cdf8905 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:54.079825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1242/2507] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:23:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:23:56.117802 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63f2ee-96f8-436a-a164-23216adc660d None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:56.118740 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63f2ee-96f8-436a-a164-23216adc660d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:56.118740 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63f2ee-96f8-436a-a164-23216adc660d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:56.155865 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-4a63f2ee-96f8-436a-a164-23216adc660d None None] MFA Rules not processed for user `5a7f399240ee4003b0e72fd34d96f47b`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:23:56.158112 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-4a63f2ee-96f8-436a-a164-23216adc660d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:56.279836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b184d6f-7eeb-4604-854c-77a17cdf8905 service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:56.281471 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3b184d6f-7eeb-4604-854c-77a17cdf8905 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:23:56.288542 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1266/2508] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:23:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:23:56.299576 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b184d6f-7eeb-4604-854c-77a17cdf8905 service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:56.304954 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd86024-c3fe-49f2-834a-3d6d8f64a133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:56.305587 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd86024-c3fe-49f2-834a-3d6d8f64a133 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:56.305872 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd86024-c3fe-49f2-834a-3d6d8f64a133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:56.306743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd86024-c3fe-49f2-834a-3d6d8f64a133 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:56.307445 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd86024-c3fe-49f2-834a-3d6d8f64a133 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:56.309399 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd86024-c3fe-49f2-834a-3d6d8f64a133 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:56.361310 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1243/2509] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:23:56 2022] POST /identity/v3/projects => generated 371 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:56.365321 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63f2ee-96f8-436a-a164-23216adc660d None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:56.384130 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63f2ee-96f8-436a-a164-23216adc660d None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:56.392082 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:23:56.493469 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c486c7f-3a7f-402d-9275-9161d96a0a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:56.495681 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c486c7f-3a7f-402d-9275-9161d96a0a6a admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:56.496243 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c486c7f-3a7f-402d-9275-9161d96a0a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:56.496707 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c486c7f-3a7f-402d-9275-9161d96a0a6a admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:56.497586 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c486c7f-3a7f-402d-9275-9161d96a0a6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:56.499755 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c486c7f-3a7f-402d-9275-9161d96a0a6a admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:56.529673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1267/2510] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:23:56 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:23:56.534040 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd86024-c3fe-49f2-834a-3d6d8f64a133 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:56.546435 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd86024-c3fe-49f2-834a-3d6d8f64a133 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:56.552552 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d89b6-2778-4f87-91fb-08e7c367ef11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:56.553464 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d89b6-2778-4f87-91fb-08e7c367ef11 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:56.553962 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d89b6-2778-4f87-91fb-08e7c367ef11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:56.554473 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d89b6-2778-4f87-91fb-08e7c367ef11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:56.555408 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d89b6-2778-4f87-91fb-08e7c367ef11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:56.557962 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d89b6-2778-4f87-91fb-08e7c367ef11 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:56.583997 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1244/2511] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:56 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:56.585188 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c486c7f-3a7f-402d-9275-9161d96a0a6a admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:56.615521 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c486c7f-3a7f-402d-9275-9161d96a0a6a admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:56.621955 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c11015-5cd7-430c-82d4-4ca2292e713c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:56.622808 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c11015-5cd7-430c-82d4-4ca2292e713c admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:56.623198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c11015-5cd7-430c-82d4-4ca2292e713c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:56.623565 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c11015-5cd7-430c-82d4-4ca2292e713c admin admin] PATH_INFO: `/v3/projects/a0aadeb9f2a64ce6a1cfb9fd546ac7eb/users/a55596588f0f4bef9f953bd1ec3c4a87/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:56.661415 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c11015-5cd7-430c-82d4-4ca2292e713c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0aadeb9f2a64ce6a1cfb9fd546ac7eb, user_id=a55596588f0f4bef9f953bd1ec3c4a87, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:56.664425 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c11015-5cd7-430c-82d4-4ca2292e713c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:56.693054 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1268/2512] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:23:56 2022] PUT /identity/v3/projects/a0aadeb9f2a64ce6a1cfb9fd546ac7eb/users/a55596588f0f4bef9f953bd1ec3c4a87/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:23:56.694324 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d89b6-2778-4f87-91fb-08e7c367ef11 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:56.722157 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d89b6-2778-4f87-91fb-08e7c367ef11 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:56.818104 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3241f0e-7300-457e-bac3-214e3d90569e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:56.818904 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3241f0e-7300-457e-bac3-214e3d90569e admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:56.819280 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3241f0e-7300-457e-bac3-214e3d90569e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:56.819675 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3241f0e-7300-457e-bac3-214e3d90569e admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:56.820739 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3241f0e-7300-457e-bac3-214e3d90569e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:56.826632 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3241f0e-7300-457e-bac3-214e3d90569e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:56.855665 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1245/2513] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:23:56 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:23:56.859507 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c11015-5cd7-430c-82d4-4ca2292e713c admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:23:56.870809 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c11015-5cd7-430c-82d4-4ca2292e713c admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:23:56.878259 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7fec9-45ad-46b5-8504-df6e1d3a1ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:23:56.879541 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7fec9-45ad-46b5-8504-df6e1d3a1ec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:23:56.879935 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7fec9-45ad-46b5-8504-df6e1d3a1ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:23:56.880081 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7fec9-45ad-46b5-8504-df6e1d3a1ec2 admin admin] PATH_INFO: `/v3/projects/a0aadeb9f2a64ce6a1cfb9fd546ac7eb/users/a55596588f0f4bef9f953bd1ec3c4a87/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:23:56.882191 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e7fec9-45ad-46b5-8504-df6e1d3a1ec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0aadeb9f2a64ce6a1cfb9fd546ac7eb, user_id=a55596588f0f4bef9f953bd1ec3c4a87, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:23:56.885613 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e7fec9-45ad-46b5-8504-df6e1d3a1ec2 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:23:56.897329 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1269/2514] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:23:56 2022] PUT /identity/v3/projects/a0aadeb9f2a64ce6a1cfb9fd546ac7eb/users/a55596588f0f4bef9f953bd1ec3c4a87/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:24:00.967971 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace73abb-b8f0-48dc-a269-5df5cacc52cf None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:00.968451 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace73abb-b8f0-48dc-a269-5df5cacc52cf None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:00.968743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace73abb-b8f0-48dc-a269-5df5cacc52cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:01.036416 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-ace73abb-b8f0-48dc-a269-5df5cacc52cf None None] MFA Rules not processed for user `eea5b08eede4409484dc2f4305b5298a`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:24:01.037590 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ace73abb-b8f0-48dc-a269-5df5cacc52cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:24:01.217517 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1246/2515] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:24:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:24:01.219269 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7fec9-45ad-46b5-8504-df6e1d3a1ec2 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:24:01.376434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7fec9-45ad-46b5-8504-df6e1d3a1ec2 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:24:01.393451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce825d93-c761-4981-a61b-42e3aa68a613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:24:01.394321 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce825d93-c761-4981-a61b-42e3aa68a613 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:01.394613 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce825d93-c761-4981-a61b-42e3aa68a613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:01.394894 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce825d93-c761-4981-a61b-42e3aa68a613 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:01.395896 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce825d93-c761-4981-a61b-42e3aa68a613 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:24:01.397813 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce825d93-c761-4981-a61b-42e3aa68a613 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:24:01.484062 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1270/2516] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:24:01 2022] POST /identity/v3/projects => generated 379 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:24:01.488379 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace73abb-b8f0-48dc-a269-5df5cacc52cf None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:24:01.519350 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace73abb-b8f0-48dc-a269-5df5cacc52cf None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:24:01.686476 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:24:01.727272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d39536-3263-4871-bad4-e6c66ac5d4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:24:01.731840 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d39536-3263-4871-bad4-e6c66ac5d4bc admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:01.733020 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d39536-3263-4871-bad4-e6c66ac5d4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:01.734259 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d39536-3263-4871-bad4-e6c66ac5d4bc admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:01.736165 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d39536-3263-4871-bad4-e6c66ac5d4bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:24:01.742514 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d39536-3263-4871-bad4-e6c66ac5d4bc admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:24:01.814819 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1247/2517] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:24:01 2022] POST /identity/v3/users => generated 296 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:24:01.818567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce825d93-c761-4981-a61b-42e3aa68a613 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:24:01.844349 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce825d93-c761-4981-a61b-42e3aa68a613 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:24:01.868079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc00067-30b5-4e3f-9256-285aa267b5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:24:01.868795 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc00067-30b5-4e3f-9256-285aa267b5bb admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:01.869676 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc00067-30b5-4e3f-9256-285aa267b5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:01.869676 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc00067-30b5-4e3f-9256-285aa267b5bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:01.871025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc00067-30b5-4e3f-9256-285aa267b5bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:24:01.872665 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc00067-30b5-4e3f-9256-285aa267b5bb admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:24:01.909443 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d39536-3263-4871-bad4-e6c66ac5d4bc admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:24:01.910342 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1271/2518] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:24:01.928274 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d39536-3263-4871-bad4-e6c66ac5d4bc admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:24:01.953277 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a744b86c-7ce8-43a1-810e-a039b6b70831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:24:01.954104 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a744b86c-7ce8-43a1-810e-a039b6b70831 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:01.954344 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a744b86c-7ce8-43a1-810e-a039b6b70831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:01.954628 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a744b86c-7ce8-43a1-810e-a039b6b70831 admin admin] PATH_INFO: `/v3/projects/f2f7e221014e4094831675d8e385da46/users/fbff1c3ef2cf437b995e1096c360e894/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:01.990914 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a744b86c-7ce8-43a1-810e-a039b6b70831 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2f7e221014e4094831675d8e385da46, user_id=fbff1c3ef2cf437b995e1096c360e894, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:24:01.995110 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a744b86c-7ce8-43a1-810e-a039b6b70831 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:24:02.027524 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1248/2519] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:24:01 2022] PUT /identity/v3/projects/f2f7e221014e4094831675d8e385da46/users/fbff1c3ef2cf437b995e1096c360e894/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:24:02.038639 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc00067-30b5-4e3f-9256-285aa267b5bb admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:24:02.058682 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc00067-30b5-4e3f-9256-285aa267b5bb admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:24:02.145136 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:24:02.222683 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8025041-37df-424d-a551-effff402613b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:24:02.223968 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8025041-37df-424d-a551-effff402613b admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:02.224404 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8025041-37df-424d-a551-effff402613b admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:02.225101 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8025041-37df-424d-a551-effff402613b admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:02.225990 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8025041-37df-424d-a551-effff402613b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:24:02.227767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8025041-37df-424d-a551-effff402613b admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:24:02.281794 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1272/2520] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:24:02.282783 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a744b86c-7ce8-43a1-810e-a039b6b70831 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:24:02.298774 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a744b86c-7ce8-43a1-810e-a039b6b70831 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:24:02.327465 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb4592a-e185-4dc7-99e8-f8545558a369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:24:02.328778 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4592a-e185-4dc7-99e8-f8545558a369 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:02.329109 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4592a-e185-4dc7-99e8-f8545558a369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:02.329392 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4592a-e185-4dc7-99e8-f8545558a369 admin admin] PATH_INFO: `/v3/projects/f2f7e221014e4094831675d8e385da46/users/fbff1c3ef2cf437b995e1096c360e894/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:02.346707 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb4592a-e185-4dc7-99e8-f8545558a369 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2f7e221014e4094831675d8e385da46, user_id=fbff1c3ef2cf437b995e1096c360e894, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:24:02.352009 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb4592a-e185-4dc7-99e8-f8545558a369 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:24:02.384892 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1249/2521] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:24:02 2022] PUT /identity/v3/projects/f2f7e221014e4094831675d8e385da46/users/fbff1c3ef2cf437b995e1096c360e894/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:24:06.158482 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c055067-29ed-47ea-850a-2b94c95a1afa None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:06.158938 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c055067-29ed-47ea-850a-2b94c95a1afa None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:06.159187 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c055067-29ed-47ea-850a-2b94c95a1afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:06.204060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-2c055067-29ed-47ea-850a-2b94c95a1afa None None] MFA Rules not processed for user `a55596588f0f4bef9f953bd1ec3c4a87`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:24:06.205310 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2c055067-29ed-47ea-850a-2b94c95a1afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:24:06.345658 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1273/2522] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:24:06 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:24:06.363047 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb4592a-e185-4dc7-99e8-f8545558a369 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:24:06.365511 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-8bb4592a-e185-4dc7-99e8-f8545558a369 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:24:06.489587 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb4592a-e185-4dc7-99e8-f8545558a369 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:24:06.502253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6895524a-0bec-4d93-9bae-95d0b547fb89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:24:06.504557 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895524a-0bec-4d93-9bae-95d0b547fb89 service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:06.505022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895524a-0bec-4d93-9bae-95d0b547fb89 service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:06.505552 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895524a-0bec-4d93-9bae-95d0b547fb89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:06.586149 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6895524a-0bec-4d93-9bae-95d0b547fb89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:24:06.587912 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6895524a-0bec-4d93-9bae-95d0b547fb89 service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:24:06.598006 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1250/2523] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:24:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:24:07.203722 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c055067-29ed-47ea-850a-2b94c95a1afa None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:24:07.206457 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2c055067-29ed-47ea-850a-2b94c95a1afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:24:07.369351 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c055067-29ed-47ea-850a-2b94c95a1afa None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:24:07.374709 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357fa243-5bbd-4f2e-91bd-e87781203b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:24:07.375438 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357fa243-5bbd-4f2e-91bd-e87781203b1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:07.375982 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357fa243-5bbd-4f2e-91bd-e87781203b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:07.376467 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357fa243-5bbd-4f2e-91bd-e87781203b1c admin admin] PATH_INFO: `/v3/users/6a601cf1725e449f80eee24e95a757ea` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:07.377885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357fa243-5bbd-4f2e-91bd-e87781203b1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a601cf1725e449f80eee24e95a757ea)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:24:07.380645 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357fa243-5bbd-4f2e-91bd-e87781203b1c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:24:07.515593 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-357fa243-5bbd-4f2e-91bd-e87781203b1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a601cf1725e449f80eee24e95a757ea'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:24:07.516558 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-357fa243-5bbd-4f2e-91bd-e87781203b1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a601cf1725e449f80eee24e95a757ea'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:24:07.517470 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-357fa243-5bbd-4f2e-91bd-e87781203b1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a601cf1725e449f80eee24e95a757ea'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:24:07.540923 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:24:07.592769 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-357fa243-5bbd-4f2e-91bd-e87781203b1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a601cf1725e449f80eee24e95a757ea'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:24:07.604603 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-357fa243-5bbd-4f2e-91bd-e87781203b1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a601cf1725e449f80eee24e95a757ea'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:24:07.641283 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1274/2524] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:24:07 2022] DELETE /identity/v3/users/6a601cf1725e449f80eee24e95a757ea => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:24:07.648416 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9619749-a4e8-4a41-a34e-a175c6f4d6d2 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:07.648894 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9619749-a4e8-4a41-a34e-a175c6f4d6d2 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:07.648999 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9619749-a4e8-4a41-a34e-a175c6f4d6d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:07.692807 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-c9619749-a4e8-4a41-a34e-a175c6f4d6d2 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:24:07.694171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c9619749-a4e8-4a41-a34e-a175c6f4d6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:24:07.796577 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:24:07.852480 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1251/2525] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:24:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:24:07.873304 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357fa243-5bbd-4f2e-91bd-e87781203b1c admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:24:07.875070 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-357fa243-5bbd-4f2e-91bd-e87781203b1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:24:08.013918 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357fa243-5bbd-4f2e-91bd-e87781203b1c admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:24:08.020350 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46eb4ef-2ea6-4c37-b74c-8a654566f5b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:24:08.021447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46eb4ef-2ea6-4c37-b74c-8a654566f5b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:08.021868 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46eb4ef-2ea6-4c37-b74c-8a654566f5b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:08.022422 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46eb4ef-2ea6-4c37-b74c-8a654566f5b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:08.035958 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46eb4ef-2ea6-4c37-b74c-8a654566f5b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:24:08.037825 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46eb4ef-2ea6-4c37-b74c-8a654566f5b1 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:24:08.051944 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1275/2526] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:24:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:24:08.328611 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9619749-a4e8-4a41-a34e-a175c6f4d6d2 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:24:08.330980 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c9619749-a4e8-4a41-a34e-a175c6f4d6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:24:08.362238 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9619749-a4e8-4a41-a34e-a175c6f4d6d2 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:24:08.380151 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45346c0f-dcde-46bf-9950-4c85a20f9490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:24:08.383876 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45346c0f-dcde-46bf-9950-4c85a20f9490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:08.384845 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45346c0f-dcde-46bf-9950-4c85a20f9490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:08.385140 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45346c0f-dcde-46bf-9950-4c85a20f9490 admin admin] PATH_INFO: `/v3/users/ded747a48fec476bb98a8cf2a9c871d7` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:08.388591 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45346c0f-dcde-46bf-9950-4c85a20f9490 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ded747a48fec476bb98a8cf2a9c871d7)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:24:08.395453 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45346c0f-dcde-46bf-9950-4c85a20f9490 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:24:08.571513 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-45346c0f-dcde-46bf-9950-4c85a20f9490 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ded747a48fec476bb98a8cf2a9c871d7'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:24:08.575673 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-45346c0f-dcde-46bf-9950-4c85a20f9490 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ded747a48fec476bb98a8cf2a9c871d7'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:24:08.616093 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-45346c0f-dcde-46bf-9950-4c85a20f9490 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ded747a48fec476bb98a8cf2a9c871d7'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:24:08.669327 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-45346c0f-dcde-46bf-9950-4c85a20f9490 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ded747a48fec476bb98a8cf2a9c871d7'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:24:08.705446 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-45346c0f-dcde-46bf-9950-4c85a20f9490 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ded747a48fec476bb98a8cf2a9c871d7'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:24:08.731317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1252/2527] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:24:08 2022] DELETE /identity/v3/users/ded747a48fec476bb98a8cf2a9c871d7 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:24:08.736002 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064e75b-276a-4789-95ab-b573d1474880 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:08.736496 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064e75b-276a-4789-95ab-b573d1474880 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:08.736736 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064e75b-276a-4789-95ab-b573d1474880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:08.780379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-1064e75b-276a-4789-95ab-b573d1474880 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:24:08.783137 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-1064e75b-276a-4789-95ab-b573d1474880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:24:08.926436 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1276/2528] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:24:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:24:09.003113 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45346c0f-dcde-46bf-9950-4c85a20f9490 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:24:09.013471 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-45346c0f-dcde-46bf-9950-4c85a20f9490 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:24:09.031394 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4184045-dbc5-459e-96da-ec8264444242 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:09.031824 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4184045-dbc5-459e-96da-ec8264444242 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:09.032205 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4184045-dbc5-459e-96da-ec8264444242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:09.080334 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-c4184045-dbc5-459e-96da-ec8264444242 None None] MFA Rules not processed for user `5a7f399240ee4003b0e72fd34d96f47b`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:24:09.083935 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c4184045-dbc5-459e-96da-ec8264444242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:24:09.184328 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45346c0f-dcde-46bf-9950-4c85a20f9490 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:24:09.193328 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb185d-acbd-4bcb-9d6d-f289c41ed684 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:24:09.195769 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb185d-acbd-4bcb-9d6d-f289c41ed684 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:09.196241 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb185d-acbd-4bcb-9d6d-f289c41ed684 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:09.197465 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb185d-acbd-4bcb-9d6d-f289c41ed684 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:09.220542 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1277/2529] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:24:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:24:09.220542 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4184045-dbc5-459e-96da-ec8264444242 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:24:09.222850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c4184045-dbc5-459e-96da-ec8264444242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:24:09.233622 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb185d-acbd-4bcb-9d6d-f289c41ed684 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:24:09.236318 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb185d-acbd-4bcb-9d6d-f289c41ed684 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:24:09.253994 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4184045-dbc5-459e-96da-ec8264444242 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:24:09.260229 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1253/2530] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:24:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:24:09.279038 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc0ebea-74c3-4cf0-a1b4-509d4b93d699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:24:09.280101 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0ebea-74c3-4cf0-a1b4-509d4b93d699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:09.282712 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0ebea-74c3-4cf0-a1b4-509d4b93d699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:09.283809 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0ebea-74c3-4cf0-a1b4-509d4b93d699 admin admin] PATH_INFO: `/v3/projects/697391cefcd64122af7944252141cfc3` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:09.289543 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc0ebea-74c3-4cf0-a1b4-509d4b93d699 admin admin] RBAC: Authorizing `identity:delete_project(project_id=697391cefcd64122af7944252141cfc3)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:24:09.292818 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc0ebea-74c3-4cf0-a1b4-509d4b93d699 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:24:09.370497 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:24:09.466979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0cc0ebea-74c3-4cf0-a1b4-509d4b93d699 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '697391cefcd64122af7944252141cfc3', 'request_id': 'req-0cc0ebea-74c3-4cf0-a1b4-509d4b93d699'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:24:09.493911 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0cc0ebea-74c3-4cf0-a1b4-509d4b93d699 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '697391cefcd64122af7944252141cfc3', 'request_id': 'req-0cc0ebea-74c3-4cf0-a1b4-509d4b93d699'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:24:09.515403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0cc0ebea-74c3-4cf0-a1b4-509d4b93d699 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '697391cefcd64122af7944252141cfc3', 'request_id': 'req-0cc0ebea-74c3-4cf0-a1b4-509d4b93d699'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:24:09.555195 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0cc0ebea-74c3-4cf0-a1b4-509d4b93d699 admin admin] The token cache is being invalidate because project 697391cefcd64122af7944252141cfc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:24:09.558026 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0cc0ebea-74c3-4cf0-a1b4-509d4b93d699 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:24:09.559122 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0cc0ebea-74c3-4cf0-a1b4-509d4b93d699 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:24:09.565298 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1278/2531] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:24:09 2022] DELETE /identity/v3/projects/697391cefcd64122af7944252141cfc3 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:24:09.570621 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc8853-c7d7-4955-a957-dd3892c8a395 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:09.570922 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc8853-c7d7-4955-a957-dd3892c8a395 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:09.571285 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc8853-c7d7-4955-a957-dd3892c8a395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:09.625622 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-f3cc8853-c7d7-4955-a957-dd3892c8a395 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:24:09.632428 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f3cc8853-c7d7-4955-a957-dd3892c8a395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:24:09.756426 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:24:09.784958 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514d1fa-11cf-425a-b04d-6af15f61c28b None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:09.785439 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514d1fa-11cf-425a-b04d-6af15f61c28b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:09.785727 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514d1fa-11cf-425a-b04d-6af15f61c28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:09.786432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1254/2532] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:24:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:24:09.839075 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-d514d1fa-11cf-425a-b04d-6af15f61c28b None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:24:09.842712 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d514d1fa-11cf-425a-b04d-6af15f61c28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:24:09.874441 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1279/2533] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:24:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:24:09.876941 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc8853-c7d7-4955-a957-dd3892c8a395 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:24:09.900293 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc8853-c7d7-4955-a957-dd3892c8a395 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:24:09.913831 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d63392-233c-40e6-b637-6ad4d4e98dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:24:09.917585 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d63392-233c-40e6-b637-6ad4d4e98dd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:09.919164 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d63392-233c-40e6-b637-6ad4d4e98dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:09.920302 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d63392-233c-40e6-b637-6ad4d4e98dd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:09.922754 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d63392-233c-40e6-b637-6ad4d4e98dd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:24:09.923915 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d63392-233c-40e6-b637-6ad4d4e98dd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:24:09.929490 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d63392-233c-40e6-b637-6ad4d4e98dd3 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:24:09.966233 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1255/2534] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:24:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:24:09.967455 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514d1fa-11cf-425a-b04d-6af15f61c28b None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:24:09.988372 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514d1fa-11cf-425a-b04d-6af15f61c28b None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:24:09.993672 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc453c-cbda-49e0-ae84-c989ccf6f7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:24:09.994585 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc453c-cbda-49e0-ae84-c989ccf6f7e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:09.995042 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc453c-cbda-49e0-ae84-c989ccf6f7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:09.995485 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc453c-cbda-49e0-ae84-c989ccf6f7e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:09.997203 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc453c-cbda-49e0-ae84-c989ccf6f7e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:24:09.999323 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc453c-cbda-49e0-ae84-c989ccf6f7e5 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:24:10.057434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1280/2535] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:24:09 2022] POST /identity/v3/projects => generated 397 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:24:10.063883 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d63392-233c-40e6-b637-6ad4d4e98dd3 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:24:10.102459 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d63392-233c-40e6-b637-6ad4d4e98dd3 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:24:10.319647 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad25884-9e81-4b38-ac10-20d52f14a997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:24:10.322594 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad25884-9e81-4b38-ac10-20d52f14a997 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:10.323114 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad25884-9e81-4b38-ac10-20d52f14a997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:10.323701 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad25884-9e81-4b38-ac10-20d52f14a997 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:10.328845 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad25884-9e81-4b38-ac10-20d52f14a997 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:24:10.336028 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad25884-9e81-4b38-ac10-20d52f14a997 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:24:10.391031 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1256/2536] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:24:10 2022] POST /identity/v3/users => generated 299 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:24:10.395010 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc453c-cbda-49e0-ae84-c989ccf6f7e5 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:24:10.414391 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc453c-cbda-49e0-ae84-c989ccf6f7e5 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:24:10.427523 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d64cba0-4385-4541-915e-eb7d65a8c93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:24:10.428174 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64cba0-4385-4541-915e-eb7d65a8c93d admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:10.428458 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64cba0-4385-4541-915e-eb7d65a8c93d admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:10.428737 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64cba0-4385-4541-915e-eb7d65a8c93d admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:10.429452 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d64cba0-4385-4541-915e-eb7d65a8c93d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:24:10.431727 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d64cba0-4385-4541-915e-eb7d65a8c93d admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:24:10.464654 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1281/2537] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:24:10.467524 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad25884-9e81-4b38-ac10-20d52f14a997 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:24:10.484157 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad25884-9e81-4b38-ac10-20d52f14a997 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:24:10.492639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbca79cf-8056-48cd-ac87-c23510cbe284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:24:10.494643 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca79cf-8056-48cd-ac87-c23510cbe284 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:10.495205 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca79cf-8056-48cd-ac87-c23510cbe284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:10.495794 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca79cf-8056-48cd-ac87-c23510cbe284 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:10.496889 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbca79cf-8056-48cd-ac87-c23510cbe284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:24:10.498768 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbca79cf-8056-48cd-ac87-c23510cbe284 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:24:10.523598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1257/2538] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:24:10.527860 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d64cba0-4385-4541-915e-eb7d65a8c93d admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:24:10.540656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d64cba0-4385-4541-915e-eb7d65a8c93d admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:24:10.559567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd94486-8310-408e-ba8c-5f4a0617ff5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:24:10.561234 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd94486-8310-408e-ba8c-5f4a0617ff5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:10.561708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd94486-8310-408e-ba8c-5f4a0617ff5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:10.562213 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd94486-8310-408e-ba8c-5f4a0617ff5f admin admin] PATH_INFO: `/v3/projects/a4ad57cf16ce485baab990d9117fb0ea/users/eee20220150749a3a0d99b8a491c535c/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:10.611499 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd94486-8310-408e-ba8c-5f4a0617ff5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4ad57cf16ce485baab990d9117fb0ea, user_id=eee20220150749a3a0d99b8a491c535c, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:24:10.619820 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd94486-8310-408e-ba8c-5f4a0617ff5f admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:24:10.648025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1282/2539] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:24:10 2022] PUT /identity/v3/projects/a4ad57cf16ce485baab990d9117fb0ea/users/eee20220150749a3a0d99b8a491c535c/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:24:10.659975 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab435d84-d411-4530-9599-2b17915d926b None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:10.660935 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab435d84-d411-4530-9599-2b17915d926b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:10.661404 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab435d84-d411-4530-9599-2b17915d926b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:10.755812 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-ab435d84-d411-4530-9599-2b17915d926b None None] MFA Rules not processed for user `eee20220150749a3a0d99b8a491c535c`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:24:10.758508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ab435d84-d411-4530-9599-2b17915d926b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:24:10.939945 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1258/2540] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:24:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:24:10.962042 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd94486-8310-408e-ba8c-5f4a0617ff5f admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:24:10.971300 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8dd94486-8310-408e-ba8c-5f4a0617ff5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:24:11.064704 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab435d84-d411-4530-9599-2b17915d926b None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:24:11.074783 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ab435d84-d411-4530-9599-2b17915d926b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:24:11.137422 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd94486-8310-408e-ba8c-5f4a0617ff5f admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:24:11.145882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ddde54-d7f1-482a-923c-04199e95ac7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:24:11.146838 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ddde54-d7f1-482a-923c-04199e95ac7a service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:11.147188 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ddde54-d7f1-482a-923c-04199e95ac7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:11.147595 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ddde54-d7f1-482a-923c-04199e95ac7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:11.178114 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:24:11.187506 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ddde54-d7f1-482a-923c-04199e95ac7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:24:11.189911 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ddde54-d7f1-482a-923c-04199e95ac7a service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:24:11.203180 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1283/2541] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:24:10 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:24:11.235326 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab435d84-d411-4530-9599-2b17915d926b None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:24:11.247706 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0faba2-694b-4eda-89e7-fdbf92b68c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:24:11.248386 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0faba2-694b-4eda-89e7-fdbf92b68c23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:11.248859 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0faba2-694b-4eda-89e7-fdbf92b68c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:11.249275 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0faba2-694b-4eda-89e7-fdbf92b68c23 admin admin] PATH_INFO: `/v3/projects/275b579029e14dc3827ab8ccfc992890` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:11.253615 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0faba2-694b-4eda-89e7-fdbf92b68c23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=275b579029e14dc3827ab8ccfc992890)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:24:11.256462 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0faba2-694b-4eda-89e7-fdbf92b68c23 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:24:11.408038 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1a0faba2-694b-4eda-89e7-fdbf92b68c23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '275b579029e14dc3827ab8ccfc992890', 'request_id': 'req-1a0faba2-694b-4eda-89e7-fdbf92b68c23'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:24:11.440086 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1a0faba2-694b-4eda-89e7-fdbf92b68c23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '275b579029e14dc3827ab8ccfc992890', 'request_id': 'req-1a0faba2-694b-4eda-89e7-fdbf92b68c23'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:24:11.463381 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1a0faba2-694b-4eda-89e7-fdbf92b68c23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '275b579029e14dc3827ab8ccfc992890', 'request_id': 'req-1a0faba2-694b-4eda-89e7-fdbf92b68c23'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:24:11.492565 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1a0faba2-694b-4eda-89e7-fdbf92b68c23 admin admin] The token cache is being invalidate because project 275b579029e14dc3827ab8ccfc992890 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:24:11.494331 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1a0faba2-694b-4eda-89e7-fdbf92b68c23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:24:11.507007 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1a0faba2-694b-4eda-89e7-fdbf92b68c23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:24:11.511799 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1259/2542] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:24:11 2022] DELETE /identity/v3/projects/275b579029e14dc3827ab8ccfc992890 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:24:12.822724 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26987c-86fa-47f3-8330-cad199c109b1 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:12.825668 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26987c-86fa-47f3-8330-cad199c109b1 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:12.826536 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26987c-86fa-47f3-8330-cad199c109b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:12.874188 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-9d26987c-86fa-47f3-8330-cad199c109b1 None None] MFA Rules not processed for user `fbff1c3ef2cf437b995e1096c360e894`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:24:12.875659 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-9d26987c-86fa-47f3-8330-cad199c109b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:24:12.956584 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:24:13.036433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1284/2543] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:24:12 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:24:13.040263 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0faba2-694b-4eda-89e7-fdbf92b68c23 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:24:13.042333 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-1a0faba2-694b-4eda-89e7-fdbf92b68c23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:24:13.159263 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0faba2-694b-4eda-89e7-fdbf92b68c23 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:24:13.175483 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf19c8c-3535-4f30-82c7-7e9d3986fafc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:24:13.177551 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf19c8c-3535-4f30-82c7-7e9d3986fafc service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:13.178340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf19c8c-3535-4f30-82c7-7e9d3986fafc service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:13.178866 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf19c8c-3535-4f30-82c7-7e9d3986fafc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:13.183199 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-fcf19c8c-3535-4f30-82c7-7e9d3986fafc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:24:13.301956 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf19c8c-3535-4f30-82c7-7e9d3986fafc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:24:13.304365 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf19c8c-3535-4f30-82c7-7e9d3986fafc service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:24:13.317109 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1260/2544] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:24:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:24:14.148064 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e1112-d517-472a-836e-03f0a1577ccd None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:14.148630 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e1112-d517-472a-836e-03f0a1577ccd None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:14.148871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e1112-d517-472a-836e-03f0a1577ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:14.174086 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-cd6e1112-d517-472a-836e-03f0a1577ccd None None] MFA Rules not processed for user `5a69202d87094049b12765d64c17e515`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:24:14.175687 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-cd6e1112-d517-472a-836e-03f0a1577ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:24:14.269086 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1285/2545] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:24:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:24:22.347018 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf19c8c-3535-4f30-82c7-7e9d3986fafc service nova] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:24:22.348734 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-fcf19c8c-3535-4f30-82c7-7e9d3986fafc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:24:22.371544 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:24:22.423124 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf19c8c-3535-4f30-82c7-7e9d3986fafc service nova] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:24:22.427940 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e648ba-6241-4edb-83f9-a0fe3e87518d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:24:22.428590 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e648ba-6241-4edb-83f9-a0fe3e87518d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:22.428900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e648ba-6241-4edb-83f9-a0fe3e87518d admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:22.429178 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e648ba-6241-4edb-83f9-a0fe3e87518d admin admin] PATH_INFO: `/v3/users/eee20220150749a3a0d99b8a491c535c` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:22.431262 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e648ba-6241-4edb-83f9-a0fe3e87518d admin admin] RBAC: Authorizing `identity:delete_user(user_id=eee20220150749a3a0d99b8a491c535c)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:24:22.433191 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e648ba-6241-4edb-83f9-a0fe3e87518d admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:24:22.485317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-96e648ba-6241-4edb-83f9-a0fe3e87518d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eee20220150749a3a0d99b8a491c535c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:24:22.486014 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-96e648ba-6241-4edb-83f9-a0fe3e87518d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eee20220150749a3a0d99b8a491c535c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:24:22.494570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-96e648ba-6241-4edb-83f9-a0fe3e87518d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eee20220150749a3a0d99b8a491c535c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:24:22.507419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-96e648ba-6241-4edb-83f9-a0fe3e87518d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eee20220150749a3a0d99b8a491c535c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:24:22.514513 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-96e648ba-6241-4edb-83f9-a0fe3e87518d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eee20220150749a3a0d99b8a491c535c'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:24:22.520473 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1261/2546] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:24:22 2022] DELETE /identity/v3/users/eee20220150749a3a0d99b8a491c535c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:24:22.525433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5cf9b-fdc6-4864-b661-c4a89df515df None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:22.525947 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5cf9b-fdc6-4864-b661-c4a89df515df None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:22.526083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5cf9b-fdc6-4864-b661-c4a89df515df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:22.544778 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-e9e5cf9b-fdc6-4864-b661-c4a89df515df None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:24:22.546102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e9e5cf9b-fdc6-4864-b661-c4a89df515df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:24:22.609491 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1286/2547] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:24:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:24:22.619843 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e648ba-6241-4edb-83f9-a0fe3e87518d admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:24:22.621246 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-96e648ba-6241-4edb-83f9-a0fe3e87518d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:24:22.681205 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e648ba-6241-4edb-83f9-a0fe3e87518d admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:24:22.685607 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed79778-084e-45bb-91b2-8d38ac31054c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:24:22.686267 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed79778-084e-45bb-91b2-8d38ac31054c service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:22.686545 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed79778-084e-45bb-91b2-8d38ac31054c service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:22.686817 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed79778-084e-45bb-91b2-8d38ac31054c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:22.697400 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed79778-084e-45bb-91b2-8d38ac31054c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:24:22.699065 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed79778-084e-45bb-91b2-8d38ac31054c service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:24:22.709208 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1262/2548] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:24:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:24:22.990696 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e5cf9b-fdc6-4864-b661-c4a89df515df None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:24:22.995194 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e9e5cf9b-fdc6-4864-b661-c4a89df515df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:24:23.019893 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e5cf9b-fdc6-4864-b661-c4a89df515df None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:24:23.025070 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8d138-6b09-4ea1-9aa3-1f778d866f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:24:23.025735 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8d138-6b09-4ea1-9aa3-1f778d866f12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:23.026257 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8d138-6b09-4ea1-9aa3-1f778d866f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:23.026538 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8d138-6b09-4ea1-9aa3-1f778d866f12 admin admin] PATH_INFO: `/v3/projects/a4ad57cf16ce485baab990d9117fb0ea` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:23.027751 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f8d138-6b09-4ea1-9aa3-1f778d866f12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4ad57cf16ce485baab990d9117fb0ea)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:24:23.029672 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f8d138-6b09-4ea1-9aa3-1f778d866f12 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:24:23.077506 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c1f8d138-6b09-4ea1-9aa3-1f778d866f12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4ad57cf16ce485baab990d9117fb0ea', 'request_id': 'req-c1f8d138-6b09-4ea1-9aa3-1f778d866f12'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:24:23.083842 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c1f8d138-6b09-4ea1-9aa3-1f778d866f12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4ad57cf16ce485baab990d9117fb0ea', 'request_id': 'req-c1f8d138-6b09-4ea1-9aa3-1f778d866f12'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:24:23.090688 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c1f8d138-6b09-4ea1-9aa3-1f778d866f12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4ad57cf16ce485baab990d9117fb0ea', 'request_id': 'req-c1f8d138-6b09-4ea1-9aa3-1f778d866f12'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:24:23.099807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:24:23.107671 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c1f8d138-6b09-4ea1-9aa3-1f778d866f12 admin admin] The token cache is being invalidate because project a4ad57cf16ce485baab990d9117fb0ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:24:23.108027 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c1f8d138-6b09-4ea1-9aa3-1f778d866f12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:24:23.108936 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c1f8d138-6b09-4ea1-9aa3-1f778d866f12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:24:23.110958 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1287/2549] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:24:22 2022] DELETE /identity/v3/projects/a4ad57cf16ce485baab990d9117fb0ea => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:24:32.704346 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=72,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:24:33.111990 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:24:41.157549 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8965457-81c3-4d16-8e5c-9e357a938d95 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:41.158077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8965457-81c3-4d16-8e5c-9e357a938d95 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:41.158328 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8965457-81c3-4d16-8e5c-9e357a938d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:41.177319 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-d8965457-81c3-4d16-8e5c-9e357a938d95 None None] MFA Rules not processed for user `eea5b08eede4409484dc2f4305b5298a`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:24:41.178554 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d8965457-81c3-4d16-8e5c-9e357a938d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:24:41.233940 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1263/2550] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:24:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:24:47.909811 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d51f10c-4006-408e-a755-b94b349447c4 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:47.910639 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d51f10c-4006-408e-a755-b94b349447c4 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:47.911155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d51f10c-4006-408e-a755-b94b349447c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:47.932485 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-5d51f10c-4006-408e-a755-b94b349447c4 None None] MFA Rules not processed for user `937a7097c9d84402882f73082cfa408f`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:24:47.933702 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5d51f10c-4006-408e-a755-b94b349447c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:24:47.999557 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1288/2551] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:24:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:24:50.079478 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8965457-81c3-4d16-8e5c-9e357a938d95 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:24:50.080906 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d8965457-81c3-4d16-8e5c-9e357a938d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:24:50.161135 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8965457-81c3-4d16-8e5c-9e357a938d95 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:24:50.167619 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af5e46-e9a2-4026-84bc-12e233ca2665 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c08fba2d7de42218400c5e652cc3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:24:50.168390 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af5e46-e9a2-4026-84bc-12e233ca2665 service placement] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:50.168726 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af5e46-e9a2-4026-84bc-12e233ca2665 service placement] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:50.169214 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af5e46-e9a2-4026-84bc-12e233ca2665 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:50.171877 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-80af5e46-e9a2-4026-84bc-12e233ca2665 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:24:50.186853 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af5e46-e9a2-4026-84bc-12e233ca2665 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:24:50.188630 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af5e46-e9a2-4026-84bc-12e233ca2665 service placement] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:24:50.199413 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1264/2552] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:24:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:24:57.322599 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d51f10c-4006-408e-a755-b94b349447c4 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:24:57.324040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5d51f10c-4006-408e-a755-b94b349447c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:24:57.380626 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d51f10c-4006-408e-a755-b94b349447c4 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:24:57.384743 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5e0479-7d1b-44bb-848b-7fbf440ec5b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:24:57.385356 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e0479-7d1b-44bb-848b-7fbf440ec5b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:24:57.385636 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e0479-7d1b-44bb-848b-7fbf440ec5b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:24:57.385909 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e0479-7d1b-44bb-848b-7fbf440ec5b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:24:57.387653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-6d5e0479-7d1b-44bb-848b-7fbf440ec5b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:24:57.450545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5e0479-7d1b-44bb-848b-7fbf440ec5b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:24:57.452130 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5e0479-7d1b-44bb-848b-7fbf440ec5b2 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:24:57.462943 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1289/2553] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:24:57 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:25:00.194596 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=57 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:25:03.705603 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af5e46-e9a2-4026-84bc-12e233ca2665 service placement] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:25:03.707120 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-80af5e46-e9a2-4026-84bc-12e233ca2665 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:25:03.718938 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af5e46-e9a2-4026-84bc-12e233ca2665 service placement] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:25:03.722838 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28bc490-a217-4925-ac17-23872292674c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:25:03.723448 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28bc490-a217-4925-ac17-23872292674c service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:25:03.723727 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28bc490-a217-4925-ac17-23872292674c service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:25:03.723998 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28bc490-a217-4925-ac17-23872292674c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:25:03.725721 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c28bc490-a217-4925-ac17-23872292674c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:25:03.788615 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28bc490-a217-4925-ac17-23872292674c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:25:03.790922 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28bc490-a217-4925-ac17-23872292674c service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:25:03.800754 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1265/2554] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:25:03 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:25:07.457680 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=75 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:25:13.800980 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=30 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:25:18.363597 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae931e16-239e-4794-8833-12b50f95e0bf None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:25:18.364039 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae931e16-239e-4794-8833-12b50f95e0bf None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:25:18.364144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae931e16-239e-4794-8833-12b50f95e0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:25:18.383220 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-ae931e16-239e-4794-8833-12b50f95e0bf None None] MFA Rules not processed for user `8ec07ee1efba4e8089b94f24be338f86`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:25:18.384444 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-ae931e16-239e-4794-8833-12b50f95e0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:25:18.439608 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1290/2555] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Jul 27 04:25:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:25:18.482845 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28bc490-a217-4925-ac17-23872292674c service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:25:18.484569 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c28bc490-a217-4925-ac17-23872292674c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:25:18.557935 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28bc490-a217-4925-ac17-23872292674c service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:25:18.562865 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6641c36-0c14-4af8-a864-e19cc7948c28 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57b5fa7517034a39930ce7c9d12e1875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:25:18.563492 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6641c36-0c14-4af8-a864-e19cc7948c28 service swift] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:25:18.563770 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6641c36-0c14-4af8-a864-e19cc7948c28 service swift] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:25:18.564045 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6641c36-0c14-4af8-a864-e19cc7948c28 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:25:18.574819 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6641c36-0c14-4af8-a864-e19cc7948c28 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:25:18.577124 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6641c36-0c14-4af8-a864-e19cc7948c28 service swift] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:25:18.588564 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1266/2556] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 27 04:25:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 04:25:28.432505 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=25 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:25:28.583972 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=32 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:25:34.095717 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ea3b6-0406-4ccd-a665-c9ea5e51f20b None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:25:34.096288 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ea3b6-0406-4ccd-a665-c9ea5e51f20b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:25:34.096288 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ea3b6-0406-4ccd-a665-c9ea5e51f20b None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:25:34.097637 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1291/2557] 10.222.0.46 () {54 vars in 921 bytes} [Wed Jul 27 04:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:25:34.105935 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ee002-0349-410f-a2aa-65096c2437bf None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:25:34.106592 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ee002-0349-410f-a2aa-65096c2437bf None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:25:34.110072 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ee002-0349-410f-a2aa-65096c2437bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:25:34.137317 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-9d0ee002-0349-410f-a2aa-65096c2437bf None None] MFA Rules not processed for user `9dc9ea8f89c14a2a8e3272f750647dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:25:34.138640 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9d0ee002-0349-410f-a2aa-65096c2437bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:25:34.224215 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1267/2558] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Jul 27 04:25:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:25:34.240020 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4ea3b6-0406-4ccd-a665-c9ea5e51f20b None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:25:34.242155 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-cf4ea3b6-0406-4ccd-a665-c9ea5e51f20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:25:34.312715 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4ea3b6-0406-4ccd-a665-c9ea5e51f20b None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:25:34.316360 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0547a3-6e88-48ae-9abc-282dce1e93c6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e98a41a0f304fc2bb36a18a77a2d729', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:25:34.317066 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0547a3-6e88-48ae-9abc-282dce1e93c6 service glance] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:25:34.317351 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0547a3-6e88-48ae-9abc-282dce1e93c6 service glance] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:25:34.317628 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0547a3-6e88-48ae-9abc-282dce1e93c6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:25:34.331882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0547a3-6e88-48ae-9abc-282dce1e93c6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:25:34.333382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0547a3-6e88-48ae-9abc-282dce1e93c6 service glance] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:25:34.344353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1292/2559] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:25:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:25:34.412550 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34afbab7-2a66-4362-8255-17da168d87c6 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:25:34.412837 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34afbab7-2a66-4362-8255-17da168d87c6 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:25:34.413111 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34afbab7-2a66-4362-8255-17da168d87c6 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:25:34.414586 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1268/2560] 10.222.0.46 () {54 vars in 921 bytes} [Wed Jul 27 04:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:25:34.419342 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab140524-7bba-4f5b-911d-1f674b7941c6 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:25:34.419654 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab140524-7bba-4f5b-911d-1f674b7941c6 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:25:34.419937 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab140524-7bba-4f5b-911d-1f674b7941c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:25:34.440541 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-ab140524-7bba-4f5b-911d-1f674b7941c6 None None] MFA Rules not processed for user `b4847f3736d844d78a206b7eab5f5fcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:25:34.441735 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-ab140524-7bba-4f5b-911d-1f674b7941c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:25:34.449223 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1293/2561] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Jul 27 04:25:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:25:34.460785 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34afbab7-2a66-4362-8255-17da168d87c6 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:25:34.472076 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34afbab7-2a66-4362-8255-17da168d87c6 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:25:34.475607 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307bb43-6e5f-495a-aff5-59ade9816227 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:25:34.476213 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307bb43-6e5f-495a-aff5-59ade9816227 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:25:34.476495 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307bb43-6e5f-495a-aff5-59ade9816227 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:25:34.476772 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307bb43-6e5f-495a-aff5-59ade9816227 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:25:34.484517 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2307bb43-6e5f-495a-aff5-59ade9816227 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:25:34.485938 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2307bb43-6e5f-495a-aff5-59ade9816227 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:25:34.494309 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1269/2562] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:25:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:25:34.578096 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c64454-c1a0-4df7-a7db-96c8f86f354d None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:25:34.578389 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c64454-c1a0-4df7-a7db-96c8f86f354d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:25:34.578668 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c64454-c1a0-4df7-a7db-96c8f86f354d None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:25:34.580032 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1294/2563] 10.222.0.46 () {54 vars in 921 bytes} [Wed Jul 27 04:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:25:34.584993 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e55b4-9082-427a-879a-5f4bfab7be17 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:25:34.585326 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e55b4-9082-427a-879a-5f4bfab7be17 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:25:34.585857 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e55b4-9082-427a-879a-5f4bfab7be17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:25:34.608586 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-0d4e55b4-9082-427a-879a-5f4bfab7be17 None None] MFA Rules not processed for user `9dc9ea8f89c14a2a8e3272f750647dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:25:34.609984 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0d4e55b4-9082-427a-879a-5f4bfab7be17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:25:34.620255 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1270/2564] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Jul 27 04:25:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:25:34.630735 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c64454-c1a0-4df7-a7db-96c8f86f354d None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:25:34.641779 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c64454-c1a0-4df7-a7db-96c8f86f354d None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:25:34.645377 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ee7a2-a7ee-4c74-a8a1-27690931aaf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:25:34.646048 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ee7a2-a7ee-4c74-a8a1-27690931aaf7 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:25:34.646335 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ee7a2-a7ee-4c74-a8a1-27690931aaf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:25:34.646613 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ee7a2-a7ee-4c74-a8a1-27690931aaf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:25:34.656584 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ee7a2-a7ee-4c74-a8a1-27690931aaf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:25:34.658022 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ee7a2-a7ee-4c74-a8a1-27690931aaf7 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:25:34.669258 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1295/2565] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:25:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:25:42.033505 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbbac6-bf11-49ea-ab38-aae15ffe8f53 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:25:42.034009 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbbac6-bf11-49ea-ab38-aae15ffe8f53 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:25:42.034101 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbbac6-bf11-49ea-ab38-aae15ffe8f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:25:42.054942 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-0dfbbac6-bf11-49ea-ab38-aae15ffe8f53 None None] MFA Rules not processed for user `8ec07ee1efba4e8089b94f24be338f86`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:25:42.056170 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0dfbbac6-bf11-49ea-ab38-aae15ffe8f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:25:42.063523 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1271/2566] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Jul 27 04:25:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:25:42.076417 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ee7a2-a7ee-4c74-a8a1-27690931aaf7 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:25:42.088788 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ee7a2-a7ee-4c74-a8a1-27690931aaf7 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:25:42.093885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc02727b-f8e0-43a8-8cb0-055d6d8a1b74 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57b5fa7517034a39930ce7c9d12e1875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:25:42.094531 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02727b-f8e0-43a8-8cb0-055d6d8a1b74 service swift] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:25:42.094811 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02727b-f8e0-43a8-8cb0-055d6d8a1b74 service swift] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:25:42.095094 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02727b-f8e0-43a8-8cb0-055d6d8a1b74 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:25:42.103599 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc02727b-f8e0-43a8-8cb0-055d6d8a1b74 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:25:42.105540 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc02727b-f8e0-43a8-8cb0-055d6d8a1b74 service swift] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:25:42.113987 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1296/2567] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 27 04:25:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 04:25:49.852638 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfbbac6-bf11-49ea-ab38-aae15ffe8f53 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:25:49.855905 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0dfbbac6-bf11-49ea-ab38-aae15ffe8f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:25:49.940751 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfbbac6-bf11-49ea-ab38-aae15ffe8f53 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:25:49.946138 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c2cbc3-f403-4014-b8cf-a0a615326d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:25:49.947083 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2cbc3-f403-4014-b8cf-a0a615326d55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:25:49.947529 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2cbc3-f403-4014-b8cf-a0a615326d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:25:49.947965 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2cbc3-f403-4014-b8cf-a0a615326d55 admin admin] PATH_INFO: `/v3/users/937a7097c9d84402882f73082cfa408f` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:25:49.949329 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c2cbc3-f403-4014-b8cf-a0a615326d55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=937a7097c9d84402882f73082cfa408f)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:25:49.951440 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c2cbc3-f403-4014-b8cf-a0a615326d55 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:25:50.011662 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-74c2cbc3-f403-4014-b8cf-a0a615326d55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '937a7097c9d84402882f73082cfa408f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:25:50.012471 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-74c2cbc3-f403-4014-b8cf-a0a615326d55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '937a7097c9d84402882f73082cfa408f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:25:50.023573 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-74c2cbc3-f403-4014-b8cf-a0a615326d55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '937a7097c9d84402882f73082cfa408f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:25:50.036291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:25:50.038658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-74c2cbc3-f403-4014-b8cf-a0a615326d55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '937a7097c9d84402882f73082cfa408f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:25:50.049781 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-74c2cbc3-f403-4014-b8cf-a0a615326d55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '937a7097c9d84402882f73082cfa408f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:25:50.059682 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1272/2568] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:25:49 2022] DELETE /identity/v3/users/937a7097c9d84402882f73082cfa408f => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:25:50.265757 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc02727b-f8e0-43a8-8cb0-055d6d8a1b74 service swift] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:25:50.286939 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc02727b-f8e0-43a8-8cb0-055d6d8a1b74 service swift] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:25:50.354721 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc84d024-2895-446a-a40a-6d4554f85c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:25:50.355422 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84d024-2895-446a-a40a-6d4554f85c72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:25:50.355708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84d024-2895-446a-a40a-6d4554f85c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:25:50.355995 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84d024-2895-446a-a40a-6d4554f85c72 admin admin] PATH_INFO: `/v3/projects/a4c6bc5f81ac457dbb91c5bc1a8f0d61` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:25:50.357200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc84d024-2895-446a-a40a-6d4554f85c72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4c6bc5f81ac457dbb91c5bc1a8f0d61)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:25:50.359203 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc84d024-2895-446a-a40a-6d4554f85c72 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:25:50.410473 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fc84d024-2895-446a-a40a-6d4554f85c72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4c6bc5f81ac457dbb91c5bc1a8f0d61', 'request_id': 'req-fc84d024-2895-446a-a40a-6d4554f85c72'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:25:50.416992 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fc84d024-2895-446a-a40a-6d4554f85c72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4c6bc5f81ac457dbb91c5bc1a8f0d61', 'request_id': 'req-fc84d024-2895-446a-a40a-6d4554f85c72'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:25:50.422837 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fc84d024-2895-446a-a40a-6d4554f85c72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4c6bc5f81ac457dbb91c5bc1a8f0d61', 'request_id': 'req-fc84d024-2895-446a-a40a-6d4554f85c72'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:25:50.433917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:25:50.439287 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fc84d024-2895-446a-a40a-6d4554f85c72 admin admin] The token cache is being invalidate because project a4c6bc5f81ac457dbb91c5bc1a8f0d61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:25:50.439753 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fc84d024-2895-446a-a40a-6d4554f85c72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:25:50.440871 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fc84d024-2895-446a-a40a-6d4554f85c72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:25:50.442710 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1297/2569] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:25:50 2022] DELETE /identity/v3/projects/a4c6bc5f81ac457dbb91c5bc1a8f0d61 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:26:00.053327 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:DELETE=1,keystone:SELECT=6 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:26:01.891573 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953825b0-ba39-427f-9600-c4d30a796536 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:26:01.892171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953825b0-ba39-427f-9600-c4d30a796536 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:26:01.892171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953825b0-ba39-427f-9600-c4d30a796536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:26:01.910721 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-953825b0-ba39-427f-9600-c4d30a796536 None None] MFA Rules not processed for user `5a69202d87094049b12765d64c17e515`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:26:01.911921 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-953825b0-ba39-427f-9600-c4d30a796536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:26:01.967889 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1273/2570] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:26:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:26:06.043678 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a99ee5-47d4-43bf-9f78-1f684b9fb8c4 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:26:06.044504 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a99ee5-47d4-43bf-9f78-1f684b9fb8c4 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:26:06.044947 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a99ee5-47d4-43bf-9f78-1f684b9fb8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:26:06.068009 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-02a99ee5-47d4-43bf-9f78-1f684b9fb8c4 None None] MFA Rules not processed for user `8ec07ee1efba4e8089b94f24be338f86`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:26:06.069256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-02a99ee5-47d4-43bf-9f78-1f684b9fb8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:26:06.130651 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1298/2571] 10.222.0.46 () {58 vars in 967 bytes} [Wed Jul 27 04:26:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:26:06.144378 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953825b0-ba39-427f-9600-c4d30a796536 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:26:06.145881 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-953825b0-ba39-427f-9600-c4d30a796536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:26:06.217282 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953825b0-ba39-427f-9600-c4d30a796536 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:26:06.222422 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c759da5-325c-44a0-b453-57822a42e202 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57b5fa7517034a39930ce7c9d12e1875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:26:06.223166 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c759da5-325c-44a0-b453-57822a42e202 service swift] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:26:06.223451 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c759da5-325c-44a0-b453-57822a42e202 service swift] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:26:06.223727 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c759da5-325c-44a0-b453-57822a42e202 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:26:06.232141 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c759da5-325c-44a0-b453-57822a42e202 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:26:06.233744 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c759da5-325c-44a0-b453-57822a42e202 service swift] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:26:06.242462 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1274/2572] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 27 04:26:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 27 04:26:16.122700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=25 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:26:16.238921 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=57 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:26:39.625711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a99ee5-47d4-43bf-9f78-1f684b9fb8c4 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:26:39.627082 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-02a99ee5-47d4-43bf-9f78-1f684b9fb8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:26:39.684290 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a99ee5-47d4-43bf-9f78-1f684b9fb8c4 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:26:39.687867 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f032f5-fb77-46fe-820e-7ae10979b8c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:26:39.688487 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f032f5-fb77-46fe-820e-7ae10979b8c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:26:39.688770 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f032f5-fb77-46fe-820e-7ae10979b8c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:26:39.689046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f032f5-fb77-46fe-820e-7ae10979b8c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:26:39.690815 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-20f032f5-fb77-46fe-820e-7ae10979b8c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:26:39.760093 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f032f5-fb77-46fe-820e-7ae10979b8c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:26:39.762176 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f032f5-fb77-46fe-820e-7ae10979b8c9 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:26:39.773386 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1299/2573] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:26:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:26:40.947029 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c759da5-325c-44a0-b453-57822a42e202 service swift] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:26:40.949130 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9c759da5-325c-44a0-b453-57822a42e202 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:26:40.960874 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c759da5-325c-44a0-b453-57822a42e202 service swift] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:26:40.964519 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe3d2d9-80a5-4eb8-aeb1-08aac11b9963 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:26:40.965081 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3d2d9-80a5-4eb8-aeb1-08aac11b9963 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:26:40.965361 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3d2d9-80a5-4eb8-aeb1-08aac11b9963 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:26:40.965633 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3d2d9-80a5-4eb8-aeb1-08aac11b9963 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:26:40.967381 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3fe3d2d9-80a5-4eb8-aeb1-08aac11b9963 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:26:40.980965 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe3d2d9-80a5-4eb8-aeb1-08aac11b9963 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:26:40.982428 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe3d2d9-80a5-4eb8-aeb1-08aac11b9963 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:26:40.993213 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1275/2574] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:26:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:26:46.882820 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f032f5-fb77-46fe-820e-7ae10979b8c9 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:26:46.884280 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-20f032f5-fb77-46fe-820e-7ae10979b8c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:26:46.898673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f032f5-fb77-46fe-820e-7ae10979b8c9 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:26:46.905416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd8126e-02d2-4026-9bc3-1c67b39f0b5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:26:46.906339 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8126e-02d2-4026-9bc3-1c67b39f0b5c service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:26:46.906631 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8126e-02d2-4026-9bc3-1c67b39f0b5c service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:26:46.907869 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8126e-02d2-4026-9bc3-1c67b39f0b5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:26:46.910262 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-4dd8126e-02d2-4026-9bc3-1c67b39f0b5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:26:46.921425 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd8126e-02d2-4026-9bc3-1c67b39f0b5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:26:46.923055 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd8126e-02d2-4026-9bc3-1c67b39f0b5c service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:26:46.932207 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1300/2575] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:26:46 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:26:50.242635 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2adb3-a291-4e85-9086-f31ddf231f17 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:26:50.243155 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2adb3-a291-4e85-9086-f31ddf231f17 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:26:50.243155 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2adb3-a291-4e85-9086-f31ddf231f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:26:50.263172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-dcd2adb3-a291-4e85-9086-f31ddf231f17 None None] MFA Rules not processed for user `0eaa342a6bc84529a5d252c7a9a161ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:26:50.265351 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-dcd2adb3-a291-4e85-9086-f31ddf231f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:26:50.330744 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1276/2576] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:26:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:26:50.637042 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd8126e-02d2-4026-9bc3-1c67b39f0b5c service nova] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:26:50.646412 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd8126e-02d2-4026-9bc3-1c67b39f0b5c service nova] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:26:50.649950 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334d427-985b-415c-8fee-90f7632ca1e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:26:50.650592 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334d427-985b-415c-8fee-90f7632ca1e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:26:50.650872 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334d427-985b-415c-8fee-90f7632ca1e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:26:50.651145 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334d427-985b-415c-8fee-90f7632ca1e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:26:50.653620 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b334d427-985b-415c-8fee-90f7632ca1e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:26:50.662929 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334d427-985b-415c-8fee-90f7632ca1e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:26:50.664375 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334d427-985b-415c-8fee-90f7632ca1e8 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:26:50.672766 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1301/2577] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:26:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:26:50.985717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2adb3-a291-4e85-9086-f31ddf231f17 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:26:50.987195 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-dcd2adb3-a291-4e85-9086-f31ddf231f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:26:51.066812 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2adb3-a291-4e85-9086-f31ddf231f17 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:26:51.072253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041de034-74c2-4260-a6cd-8bdadb122cfe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c08fba2d7de42218400c5e652cc3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:26:51.072876 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041de034-74c2-4260-a6cd-8bdadb122cfe service placement] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:26:51.073712 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041de034-74c2-4260-a6cd-8bdadb122cfe service placement] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:26:51.074024 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041de034-74c2-4260-a6cd-8bdadb122cfe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:26:51.075851 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-041de034-74c2-4260-a6cd-8bdadb122cfe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:26:51.088265 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041de034-74c2-4260-a6cd-8bdadb122cfe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:26:51.090664 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041de034-74c2-4260-a6cd-8bdadb122cfe service placement] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:26:51.100822 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1277/2578] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:26:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:26:58.835805 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334d427-985b-415c-8fee-90f7632ca1e8 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:26:58.845505 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334d427-985b-415c-8fee-90f7632ca1e8 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:26:58.849108 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c705a1-360d-427a-bb75-118e609c8cd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:26:58.849717 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c705a1-360d-427a-bb75-118e609c8cd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:26:58.850027 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c705a1-360d-427a-bb75-118e609c8cd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:26:58.850308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c705a1-360d-427a-bb75-118e609c8cd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:26:58.852055 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d6c705a1-360d-427a-bb75-118e609c8cd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:26:58.917625 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c705a1-360d-427a-bb75-118e609c8cd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:26:58.919102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c705a1-360d-427a-bb75-118e609c8cd1 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:26:58.928753 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1302/2579] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:26:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:27:01.096591 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=63 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:27:02.168383 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041de034-74c2-4260-a6cd-8bdadb122cfe service placement] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:27:02.179865 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041de034-74c2-4260-a6cd-8bdadb122cfe service placement] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:27:02.186100 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de83cd1-7b85-4e9a-af85-b2eeb40ae168 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c08fba2d7de42218400c5e652cc3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:27:02.186889 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de83cd1-7b85-4e9a-af85-b2eeb40ae168 service placement] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:27:02.187274 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de83cd1-7b85-4e9a-af85-b2eeb40ae168 service placement] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:27:02.187625 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de83cd1-7b85-4e9a-af85-b2eeb40ae168 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:27:02.189678 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-7de83cd1-7b85-4e9a-af85-b2eeb40ae168 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:27:02.210119 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de83cd1-7b85-4e9a-af85-b2eeb40ae168 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:27:02.211858 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de83cd1-7b85-4e9a-af85-b2eeb40ae168 service placement] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:27:02.231616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c705a1-360d-427a-bb75-118e609c8cd1 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:27:02.233341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d6c705a1-360d-427a-bb75-118e609c8cd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:27:02.235399 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1278/2580] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:27:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:27:02.261585 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c705a1-360d-427a-bb75-118e609c8cd1 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:27:02.266488 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac799e5-6f8b-43c4-a90d-546361d83972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:27:02.267142 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac799e5-6f8b-43c4-a90d-546361d83972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:27:02.267507 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac799e5-6f8b-43c4-a90d-546361d83972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:27:02.267782 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac799e5-6f8b-43c4-a90d-546361d83972 admin admin] PATH_INFO: `/v3/users/5a69202d87094049b12765d64c17e515` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:27:02.269049 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac799e5-6f8b-43c4-a90d-546361d83972 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a69202d87094049b12765d64c17e515)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:27:02.271083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac799e5-6f8b-43c4-a90d-546361d83972 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:27:02.273772 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:27:02.318071 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-2ac799e5-6f8b-43c4-a90d-546361d83972 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a69202d87094049b12765d64c17e515'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:27:02.318691 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-2ac799e5-6f8b-43c4-a90d-546361d83972 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a69202d87094049b12765d64c17e515'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:27:02.319300 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-2ac799e5-6f8b-43c4-a90d-546361d83972 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a69202d87094049b12765d64c17e515'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:27:02.332253 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-2ac799e5-6f8b-43c4-a90d-546361d83972 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a69202d87094049b12765d64c17e515'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:27:02.340025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-2ac799e5-6f8b-43c4-a90d-546361d83972 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a69202d87094049b12765d64c17e515'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:27:02.352733 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1303/2581] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:27:02 2022] DELETE /identity/v3/users/5a69202d87094049b12765d64c17e515 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:27:02.529042 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de83cd1-7b85-4e9a-af85-b2eeb40ae168 service placement] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:27:02.539934 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de83cd1-7b85-4e9a-af85-b2eeb40ae168 service placement] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:27:02.600699 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b682f08-6057-4266-9999-d6c0614823e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:27:02.601339 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b682f08-6057-4266-9999-d6c0614823e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:27:02.601616 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b682f08-6057-4266-9999-d6c0614823e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:27:02.601887 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b682f08-6057-4266-9999-d6c0614823e4 admin admin] PATH_INFO: `/v3/projects/5bb7f3d2bc1e4651b7f1e0ac6debd89a` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:27:02.603080 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b682f08-6057-4266-9999-d6c0614823e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bb7f3d2bc1e4651b7f1e0ac6debd89a)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:27:02.605199 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b682f08-6057-4266-9999-d6c0614823e4 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:27:02.790340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-8b682f08-6057-4266-9999-d6c0614823e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bb7f3d2bc1e4651b7f1e0ac6debd89a', 'request_id': 'req-8b682f08-6057-4266-9999-d6c0614823e4'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:27:02.797352 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-8b682f08-6057-4266-9999-d6c0614823e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bb7f3d2bc1e4651b7f1e0ac6debd89a', 'request_id': 'req-8b682f08-6057-4266-9999-d6c0614823e4'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:27:02.805663 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-8b682f08-6057-4266-9999-d6c0614823e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bb7f3d2bc1e4651b7f1e0ac6debd89a', 'request_id': 'req-8b682f08-6057-4266-9999-d6c0614823e4'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:27:02.827995 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-8b682f08-6057-4266-9999-d6c0614823e4 admin admin] The token cache is being invalidate because project 5bb7f3d2bc1e4651b7f1e0ac6debd89a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:27:02.828375 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-8b682f08-6057-4266-9999-d6c0614823e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:27:02.829097 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-8b682f08-6057-4266-9999-d6c0614823e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:27:02.831440 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1279/2582] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:27:02 2022] DELETE /identity/v3/projects/5bb7f3d2bc1e4651b7f1e0ac6debd89a => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:27:02.836120 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2c6e1-406b-4984-9b2b-d61b49264d17 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:27:02.836411 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2c6e1-406b-4984-9b2b-d61b49264d17 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:27:02.836691 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2c6e1-406b-4984-9b2b-d61b49264d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:27:02.859351 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-f6a2c6e1-406b-4984-9b2b-d61b49264d17 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:27:02.860568 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f6a2c6e1-406b-4984-9b2b-d61b49264d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:27:02.920061 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1304/2583] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:27:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:27:02.928076 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25451397-afc0-4dc0-a021-ac7e942b7d9f None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:27:02.928358 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25451397-afc0-4dc0-a021-ac7e942b7d9f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:27:02.928632 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25451397-afc0-4dc0-a021-ac7e942b7d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:27:02.948388 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-25451397-afc0-4dc0-a021-ac7e942b7d9f None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:27:02.949572 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-25451397-afc0-4dc0-a021-ac7e942b7d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:27:02.957285 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1280/2584] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:27:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:27:02.961479 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2c6e1-406b-4984-9b2b-d61b49264d17 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:27:02.971252 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2c6e1-406b-4984-9b2b-d61b49264d17 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:27:02.975901 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ed95e-8175-4560-87da-cb489d01c53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:27:02.976515 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ed95e-8175-4560-87da-cb489d01c53c admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:27:02.976801 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ed95e-8175-4560-87da-cb489d01c53c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:27:02.977078 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ed95e-8175-4560-87da-cb489d01c53c admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:27:02.977808 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ed95e-8175-4560-87da-cb489d01c53c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:27:02.978162 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ed95e-8175-4560-87da-cb489d01c53c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:27:02.980521 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ed95e-8175-4560-87da-cb489d01c53c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:27:02.992028 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1305/2585] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:27:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:27:02.996950 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25451397-afc0-4dc0-a021-ac7e942b7d9f None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:27:03.005958 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25451397-afc0-4dc0-a021-ac7e942b7d9f None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:27:03.010770 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ead9c-6cfa-4aa3-a218-9e668b43684a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:27:03.011454 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ead9c-6cfa-4aa3-a218-9e668b43684a admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:27:03.011730 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ead9c-6cfa-4aa3-a218-9e668b43684a admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:27:03.011993 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ead9c-6cfa-4aa3-a218-9e668b43684a admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:27:03.012655 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ead9c-6cfa-4aa3-a218-9e668b43684a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:27:03.014561 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ead9c-6cfa-4aa3-a218-9e668b43684a admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:27:03.043648 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1281/2586] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:27:02 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:27:03.047786 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ed95e-8175-4560-87da-cb489d01c53c admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:27:03.058882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ed95e-8175-4560-87da-cb489d01c53c admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:27:03.119520 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899a50a-3a71-4243-a477-07c4d14af78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:27:03.120860 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899a50a-3a71-4243-a477-07c4d14af78e admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:27:03.121143 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899a50a-3a71-4243-a477-07c4d14af78e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:27:03.121417 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899a50a-3a71-4243-a477-07c4d14af78e admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:27:03.122131 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c899a50a-3a71-4243-a477-07c4d14af78e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:27:03.124084 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c899a50a-3a71-4243-a477-07c4d14af78e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:27:03.140088 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:27:03.147051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1306/2587] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:27:03 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:27:03.150987 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ead9c-6cfa-4aa3-a218-9e668b43684a admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:27:03.161585 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ead9c-6cfa-4aa3-a218-9e668b43684a admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:27:03.166293 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35150a32-ed50-4219-b67b-961b9f7a6bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:27:03.167558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35150a32-ed50-4219-b67b-961b9f7a6bba admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:27:03.167926 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35150a32-ed50-4219-b67b-961b9f7a6bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:27:03.168290 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35150a32-ed50-4219-b67b-961b9f7a6bba admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:27:03.169117 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35150a32-ed50-4219-b67b-961b9f7a6bba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:27:03.170999 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35150a32-ed50-4219-b67b-961b9f7a6bba admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:27:03.181240 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1282/2588] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:27:03.186093 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899a50a-3a71-4243-a477-07c4d14af78e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:27:03.200638 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899a50a-3a71-4243-a477-07c4d14af78e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:27:03.206055 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac66d7e2-f4be-4a26-9313-7b2be20888b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:27:03.206819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66d7e2-f4be-4a26-9313-7b2be20888b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:27:03.207184 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66d7e2-f4be-4a26-9313-7b2be20888b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:27:03.207451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66d7e2-f4be-4a26-9313-7b2be20888b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:27:03.208250 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac66d7e2-f4be-4a26-9313-7b2be20888b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:27:03.209918 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac66d7e2-f4be-4a26-9313-7b2be20888b4 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:27:03.224907 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1307/2589] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:27:03.228882 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35150a32-ed50-4219-b67b-961b9f7a6bba admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:27:03.239919 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35150a32-ed50-4219-b67b-961b9f7a6bba admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:27:03.245403 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c81268-f05e-407a-9f3b-13e39e5fbddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:27:03.246224 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c81268-f05e-407a-9f3b-13e39e5fbddb admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:27:03.246594 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c81268-f05e-407a-9f3b-13e39e5fbddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:27:03.246934 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c81268-f05e-407a-9f3b-13e39e5fbddb admin admin] PATH_INFO: `/v3/projects/aa51c6088dfa44df848e339d61bac664/users/8cbc27f4433e4c48a75b216540a9022f/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:27:03.263641 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c81268-f05e-407a-9f3b-13e39e5fbddb admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa51c6088dfa44df848e339d61bac664, user_id=8cbc27f4433e4c48a75b216540a9022f, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:27:03.267707 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c81268-f05e-407a-9f3b-13e39e5fbddb admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:27:03.284235 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1283/2590] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:27:03 2022] PUT /identity/v3/projects/aa51c6088dfa44df848e339d61bac664/users/8cbc27f4433e4c48a75b216540a9022f/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:27:03.289419 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2670e-b052-44eb-8467-40a17f108746 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:27:03.289950 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2670e-b052-44eb-8467-40a17f108746 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:27:03.290085 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2670e-b052-44eb-8467-40a17f108746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:27:03.310277 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-3dc2670e-b052-44eb-8467-40a17f108746 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:27:03.311647 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-3dc2670e-b052-44eb-8467-40a17f108746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:27:03.361873 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c81268-f05e-407a-9f3b-13e39e5fbddb admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:27:03.363630 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-41c81268-f05e-407a-9f3b-13e39e5fbddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:27:03.375080 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1308/2591] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:27:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:27:03.378904 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:27:03.385310 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2670e-b052-44eb-8467-40a17f108746 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:27:03.386962 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-3dc2670e-b052-44eb-8467-40a17f108746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:27:03.425728 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c81268-f05e-407a-9f3b-13e39e5fbddb admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:27:03.429471 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ecb62-0d0d-4594-9f3e-adc831313132 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:27:03.430133 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ecb62-0d0d-4594-9f3e-adc831313132 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:27:03.430414 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ecb62-0d0d-4594-9f3e-adc831313132 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:27:03.430739 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ecb62-0d0d-4594-9f3e-adc831313132 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:27:03.432649 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-465ecb62-0d0d-4594-9f3e-adc831313132 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:27:03.442447 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465ecb62-0d0d-4594-9f3e-adc831313132 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:27:03.443868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465ecb62-0d0d-4594-9f3e-adc831313132 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:27:03.445177 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2670e-b052-44eb-8467-40a17f108746 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:27:03.449414 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a473ed7-a495-412c-9220-c11edc482f22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:27:03.450078 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a473ed7-a495-412c-9220-c11edc482f22 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:27:03.450367 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a473ed7-a495-412c-9220-c11edc482f22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:27:03.450648 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a473ed7-a495-412c-9220-c11edc482f22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:27:03.453140 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1284/2592] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:27:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 04:27:03.461823 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a473ed7-a495-412c-9220-c11edc482f22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:27:03.466010 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a473ed7-a495-412c-9220-c11edc482f22 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:27:03.474125 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1309/2593] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:27:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 04:27:07.101322 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ecb62-0d0d-4594-9f3e-adc831313132 service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:27:07.102825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-465ecb62-0d0d-4594-9f3e-adc831313132 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:27:07.245755 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ecb62-0d0d-4594-9f3e-adc831313132 service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:27:07.252513 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77de739-e023-4ab1-806b-3f5675d246e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:27:07.253279 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77de739-e023-4ab1-806b-3f5675d246e7 service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:27:07.253660 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77de739-e023-4ab1-806b-3f5675d246e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:27:07.254997 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77de739-e023-4ab1-806b-3f5675d246e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:27:07.258036 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f77de739-e023-4ab1-806b-3f5675d246e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:27:07.274570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77de739-e023-4ab1-806b-3f5675d246e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:27:07.278653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77de739-e023-4ab1-806b-3f5675d246e7 service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:27:07.308211 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1285/2594] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:27:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:27:08.231810 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505fbfb-17c9-4e18-83f5-5f730462fb1d None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:27:08.233094 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505fbfb-17c9-4e18-83f5-5f730462fb1d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:27:08.233578 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505fbfb-17c9-4e18-83f5-5f730462fb1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:27:08.274757 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-9505fbfb-17c9-4e18-83f5-5f730462fb1d None None] MFA Rules not processed for user `8cbc27f4433e4c48a75b216540a9022f`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:27:08.276189 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-9505fbfb-17c9-4e18-83f5-5f730462fb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:27:08.362098 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1310/2595] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:27:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:27:08.377518 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77de739-e023-4ab1-806b-3f5675d246e7 service nova] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:27:08.379064 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f77de739-e023-4ab1-806b-3f5675d246e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:27:08.398344 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77de739-e023-4ab1-806b-3f5675d246e7 service nova] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:27:08.404930 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9db6b6-918a-4f62-a3a6-9b45cf0b6162 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:27:08.405672 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9db6b6-918a-4f62-a3a6-9b45cf0b6162 service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:27:08.406081 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9db6b6-918a-4f62-a3a6-9b45cf0b6162 service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:27:08.406424 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9db6b6-918a-4f62-a3a6-9b45cf0b6162 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:27:08.415326 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9db6b6-918a-4f62-a3a6-9b45cf0b6162 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:27:08.417581 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9db6b6-918a-4f62-a3a6-9b45cf0b6162 service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:27:08.429048 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1286/2596] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:27:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:27:10.003756 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912dfdac-8365-446d-8bc7-70d367dde740 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:27:10.004211 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912dfdac-8365-446d-8bc7-70d367dde740 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:27:10.004306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912dfdac-8365-446d-8bc7-70d367dde740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:27:10.024124 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-912dfdac-8365-446d-8bc7-70d367dde740 None None] MFA Rules not processed for user `8cbc27f4433e4c48a75b216540a9022f`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:27:10.025609 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-912dfdac-8365-446d-8bc7-70d367dde740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:27:10.034316 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1311/2597] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:27:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:27:18.423855 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=60 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:27:20.018639 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=94 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:27:26.260700 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad1eff-0cac-4b96-bf93-4c9264bb54fc None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:27:26.261244 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad1eff-0cac-4b96-bf93-4c9264bb54fc None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:27:26.261348 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad1eff-0cac-4b96-bf93-4c9264bb54fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:27:26.281704 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-f3ad1eff-0cac-4b96-bf93-4c9264bb54fc None None] MFA Rules not processed for user `eea5b08eede4409484dc2f4305b5298a`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:27:26.283063 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f3ad1eff-0cac-4b96-bf93-4c9264bb54fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:27:26.343820 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1287/2598] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:27:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:27:33.229869 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912dfdac-8365-446d-8bc7-70d367dde740 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:27:33.247410 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912dfdac-8365-446d-8bc7-70d367dde740 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:27:33.253192 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d403410-56de-4b2c-9d3e-5c783b77c50f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:27:33.253828 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d403410-56de-4b2c-9d3e-5c783b77c50f service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:27:33.254144 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d403410-56de-4b2c-9d3e-5c783b77c50f service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:27:33.254419 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d403410-56de-4b2c-9d3e-5c783b77c50f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:27:33.256626 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7d403410-56de-4b2c-9d3e-5c783b77c50f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:27:33.270882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d403410-56de-4b2c-9d3e-5c783b77c50f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:27:33.272504 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d403410-56de-4b2c-9d3e-5c783b77c50f service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:27:33.285174 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1312/2599] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:27:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:27:36.336597 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=25 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:27:43.280968 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=6 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:27:45.360410 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad1eff-0cac-4b96-bf93-4c9264bb54fc None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:27:45.374047 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad1eff-0cac-4b96-bf93-4c9264bb54fc None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:27:45.379585 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a976b7-e5bc-4f43-bd4d-c260ff8013bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:27:45.380352 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a976b7-e5bc-4f43-bd4d-c260ff8013bc service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:27:45.380636 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a976b7-e5bc-4f43-bd4d-c260ff8013bc service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:27:45.380911 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a976b7-e5bc-4f43-bd4d-c260ff8013bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:27:45.382809 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-99a976b7-e5bc-4f43-bd4d-c260ff8013bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:27:45.396070 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a976b7-e5bc-4f43-bd4d-c260ff8013bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:27:45.397929 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a976b7-e5bc-4f43-bd4d-c260ff8013bc service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:27:45.408966 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1288/2600] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:27:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:27:55.404510 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=6 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:27:58.997166 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a3d7f-0001-4f3d-937a-be75df5bbea5 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:27:58.997634 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a3d7f-0001-4f3d-937a-be75df5bbea5 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:27:58.997735 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a3d7f-0001-4f3d-937a-be75df5bbea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:27:59.016634 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-3f9a3d7f-0001-4f3d-937a-be75df5bbea5 None None] MFA Rules not processed for user `8ec07ee1efba4e8089b94f24be338f86`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:27:59.017831 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-3f9a3d7f-0001-4f3d-937a-be75df5bbea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:27:59.073566 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1313/2601] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Jul 27 04:27:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:27:59.123227 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a976b7-e5bc-4f43-bd4d-c260ff8013bc service nova] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:27:59.124635 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-99a976b7-e5bc-4f43-bd4d-c260ff8013bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:27:59.193960 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a976b7-e5bc-4f43-bd4d-c260ff8013bc service nova] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:27:59.199735 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f39384-0b23-4b50-bdfe-208aa73801d3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57b5fa7517034a39930ce7c9d12e1875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:27:59.200353 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f39384-0b23-4b50-bdfe-208aa73801d3 service swift] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:27:59.200632 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f39384-0b23-4b50-bdfe-208aa73801d3 service swift] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:27:59.200902 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f39384-0b23-4b50-bdfe-208aa73801d3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:27:59.208845 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f39384-0b23-4b50-bdfe-208aa73801d3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:27:59.210437 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f39384-0b23-4b50-bdfe-208aa73801d3 service swift] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:27:59.219303 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1289/2602] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 27 04:27:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 04:28:09.066561 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=25 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:28:09.216441 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=32 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:28:12.119615 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a3d7f-0001-4f3d-937a-be75df5bbea5 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:28:12.121045 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-3f9a3d7f-0001-4f3d-937a-be75df5bbea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:28:12.132631 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a3d7f-0001-4f3d-937a-be75df5bbea5 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:28:12.136200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a5e76-a8c4-4dd6-822f-23118f6f5f88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:28:12.136813 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a5e76-a8c4-4dd6-822f-23118f6f5f88 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:28:12.137093 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a5e76-a8c4-4dd6-822f-23118f6f5f88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:28:12.137366 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a5e76-a8c4-4dd6-822f-23118f6f5f88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:28:12.139092 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-056a5e76-a8c4-4dd6-822f-23118f6f5f88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:28:12.148293 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a5e76-a8c4-4dd6-822f-23118f6f5f88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:28:12.149731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a5e76-a8c4-4dd6-822f-23118f6f5f88 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:28:12.157484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1314/2603] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:28:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:28:12.260379 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3ffdb-794d-4dda-8051-45711db79d96 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:28:12.260940 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3ffdb-794d-4dda-8051-45711db79d96 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:28:12.260940 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3ffdb-794d-4dda-8051-45711db79d96 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:28:12.262339 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1290/2604] 10.222.0.46 () {54 vars in 921 bytes} [Wed Jul 27 04:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:28:12.268534 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f4a91-bebe-49d6-8aee-c5deb3f7e9d0 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:28:12.268896 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f4a91-bebe-49d6-8aee-c5deb3f7e9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:28:12.269273 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f4a91-bebe-49d6-8aee-c5deb3f7e9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:28:12.289960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-041f4a91-bebe-49d6-8aee-c5deb3f7e9d0 None None] MFA Rules not processed for user `9dc9ea8f89c14a2a8e3272f750647dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:28:12.291869 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-041f4a91-bebe-49d6-8aee-c5deb3f7e9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:28:12.301239 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1315/2605] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Jul 27 04:28:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:28:12.313183 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a3ffdb-794d-4dda-8051-45711db79d96 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:28:12.315259 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b6a3ffdb-794d-4dda-8051-45711db79d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:28:12.375625 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a3ffdb-794d-4dda-8051-45711db79d96 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:28:12.379133 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb59be-8901-4f1d-87e0-38b69f73d9bd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e98a41a0f304fc2bb36a18a77a2d729', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:28:12.379835 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb59be-8901-4f1d-87e0-38b69f73d9bd service glance] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:28:12.380001 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb59be-8901-4f1d-87e0-38b69f73d9bd service glance] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:28:12.380269 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb59be-8901-4f1d-87e0-38b69f73d9bd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:28:12.390897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb59be-8901-4f1d-87e0-38b69f73d9bd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:28:12.392313 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb59be-8901-4f1d-87e0-38b69f73d9bd service glance] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:28:12.403653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1291/2606] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:28:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:28:12.472851 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f48bb3-c75f-4dfb-b008-48938128182f None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:28:12.473115 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f48bb3-c75f-4dfb-b008-48938128182f None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:28:12.473393 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f48bb3-c75f-4dfb-b008-48938128182f None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:28:12.474823 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1316/2607] 10.222.0.46 () {54 vars in 921 bytes} [Wed Jul 27 04:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:28:12.480393 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284566e4-be52-4ac9-ae3c-8ea4ee0b3051 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:28:12.480708 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284566e4-be52-4ac9-ae3c-8ea4ee0b3051 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:28:12.480983 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284566e4-be52-4ac9-ae3c-8ea4ee0b3051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:28:12.500539 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-284566e4-be52-4ac9-ae3c-8ea4ee0b3051 None None] MFA Rules not processed for user `b4847f3736d844d78a206b7eab5f5fcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:28:12.501735 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-284566e4-be52-4ac9-ae3c-8ea4ee0b3051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:28:12.508179 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1292/2608] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Jul 27 04:28:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:28:12.519900 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f48bb3-c75f-4dfb-b008-48938128182f None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:28:12.531501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f48bb3-c75f-4dfb-b008-48938128182f None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:28:12.535029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cdef56-c040-40b5-9c9e-dd431d07746c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:28:12.535635 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cdef56-c040-40b5-9c9e-dd431d07746c service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:28:12.535916 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cdef56-c040-40b5-9c9e-dd431d07746c service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:28:12.536198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cdef56-c040-40b5-9c9e-dd431d07746c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:28:12.543820 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cdef56-c040-40b5-9c9e-dd431d07746c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:28:12.545243 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cdef56-c040-40b5-9c9e-dd431d07746c service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:28:12.552865 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1317/2609] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:28:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:28:12.635469 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7143cba-98d3-4bc6-b678-12a83d39aa76 None None] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:28:12.635756 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7143cba-98d3-4bc6-b678-12a83d39aa76 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:28:12.636036 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7143cba-98d3-4bc6-b678-12a83d39aa76 None None] PATH_INFO: `/` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:28:12.637443 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1293/2610] 10.222.0.46 () {54 vars in 921 bytes} [Wed Jul 27 04:28:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 04:28:12.642697 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58fac9b-e095-4ade-ab79-c4f346941d3b None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:28:12.642985 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58fac9b-e095-4ade-ab79-c4f346941d3b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:28:12.643267 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58fac9b-e095-4ade-ab79-c4f346941d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:28:12.662964 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-a58fac9b-e095-4ade-ab79-c4f346941d3b None None] MFA Rules not processed for user `9dc9ea8f89c14a2a8e3272f750647dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:28:12.664191 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a58fac9b-e095-4ade-ab79-c4f346941d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:28:12.673707 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1318/2611] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Jul 27 04:28:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:28:12.684384 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7143cba-98d3-4bc6-b678-12a83d39aa76 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:28:12.695166 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7143cba-98d3-4bc6-b678-12a83d39aa76 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:28:12.699566 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff9917-7680-4d99-8a48-6b7e89f624a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:28:12.700156 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff9917-7680-4d99-8a48-6b7e89f624a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:28:12.700432 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff9917-7680-4d99-8a48-6b7e89f624a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:28:12.700701 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff9917-7680-4d99-8a48-6b7e89f624a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:28:12.710607 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff9917-7680-4d99-8a48-6b7e89f624a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:28:12.712003 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff9917-7680-4d99-8a48-6b7e89f624a4 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:28:12.723269 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1294/2612] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:28:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:28:14.976225 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765a69e-6597-4b4d-ad9d-0d6b341b98df None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:28:14.976711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765a69e-6597-4b4d-ad9d-0d6b341b98df None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:28:14.976819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765a69e-6597-4b4d-ad9d-0d6b341b98df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:28:14.995733 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-3765a69e-6597-4b4d-ad9d-0d6b341b98df None None] MFA Rules not processed for user `8ec07ee1efba4e8089b94f24be338f86`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:28:14.996946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-3765a69e-6597-4b4d-ad9d-0d6b341b98df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:28:15.004258 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1319/2613] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Jul 27 04:28:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:28:15.021297 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff9917-7680-4d99-8a48-6b7e89f624a4 service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:28:15.035492 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff9917-7680-4d99-8a48-6b7e89f624a4 service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:28:15.041436 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44522c0c-d79c-49ff-8c77-b1ac0d91d1a9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57b5fa7517034a39930ce7c9d12e1875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:28:15.042079 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44522c0c-d79c-49ff-8c77-b1ac0d91d1a9 service swift] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:28:15.042366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44522c0c-d79c-49ff-8c77-b1ac0d91d1a9 service swift] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:28:15.042642 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44522c0c-d79c-49ff-8c77-b1ac0d91d1a9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:28:15.051174 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44522c0c-d79c-49ff-8c77-b1ac0d91d1a9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:28:15.057258 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44522c0c-d79c-49ff-8c77-b1ac0d91d1a9 service swift] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:28:15.066260 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1295/2614] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 27 04:28:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 04:28:16.951675 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36ab14-2297-43e7-a337-2719010b3880 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:28:16.952256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36ab14-2297-43e7-a337-2719010b3880 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:28:16.952256 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36ab14-2297-43e7-a337-2719010b3880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:28:16.979681 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-ce36ab14-2297-43e7-a337-2719010b3880 None None] MFA Rules not processed for user `8cbc27f4433e4c48a75b216540a9022f`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:28:16.980936 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-ce36ab14-2297-43e7-a337-2719010b3880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:28:16.988447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1320/2615] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:28:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:28:19.389095 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44522c0c-d79c-49ff-8c77-b1ac0d91d1a9 service swift] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:28:19.401912 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44522c0c-d79c-49ff-8c77-b1ac0d91d1a9 service swift] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:28:19.407309 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273d004-132d-4a11-9aab-26e23b6e34ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:28:19.407931 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273d004-132d-4a11-9aab-26e23b6e34ff service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:28:19.408211 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273d004-132d-4a11-9aab-26e23b6e34ff service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:28:19.408481 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273d004-132d-4a11-9aab-26e23b6e34ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:28:19.410395 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-5273d004-132d-4a11-9aab-26e23b6e34ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:28:19.485355 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5273d004-132d-4a11-9aab-26e23b6e34ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:28:19.487086 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5273d004-132d-4a11-9aab-26e23b6e34ff service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:28:19.497503 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1296/2616] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:28:19 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:28:26.966498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=24 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:28:29.154570 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7f475-044a-417d-a444-36cca93a267a None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:28:29.155529 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7f475-044a-417d-a444-36cca93a267a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:28:29.155977 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7f475-044a-417d-a444-36cca93a267a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:28:29.176061 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-cdc7f475-044a-417d-a444-36cca93a267a None None] MFA Rules not processed for user `8ec07ee1efba4e8089b94f24be338f86`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:28:29.177283 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-cdc7f475-044a-417d-a444-36cca93a267a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:28:29.184567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1321/2617] 10.222.0.46 () {58 vars in 967 bytes} [Wed Jul 27 04:28:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:28:29.200557 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273d004-132d-4a11-9aab-26e23b6e34ff service nova] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:28:29.212470 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273d004-132d-4a11-9aab-26e23b6e34ff service nova] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:28:29.219100 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34fc05-ea5b-4f45-aa06-2ce4c75f9b85 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57b5fa7517034a39930ce7c9d12e1875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:28:29.219728 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34fc05-ea5b-4f45-aa06-2ce4c75f9b85 service swift] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:28:29.220079 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34fc05-ea5b-4f45-aa06-2ce4c75f9b85 service swift] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:28:29.220400 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34fc05-ea5b-4f45-aa06-2ce4c75f9b85 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:28:29.229757 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34fc05-ea5b-4f45-aa06-2ce4c75f9b85 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:28:29.231401 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34fc05-ea5b-4f45-aa06-2ce4c75f9b85 service swift] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:28:29.240064 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1297/2618] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 27 04:28:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 04:28:39.169916 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:28:39.236991 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=75 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:29:06.256428 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc7f475-044a-417d-a444-36cca93a267a None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:29:06.269786 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc7f475-044a-417d-a444-36cca93a267a None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:29:06.275263 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059230e-b41e-4bc1-8f6b-0697f9a18add service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:29:06.275886 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059230e-b41e-4bc1-8f6b-0697f9a18add service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:29:06.276168 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059230e-b41e-4bc1-8f6b-0697f9a18add service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:29:06.276465 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059230e-b41e-4bc1-8f6b-0697f9a18add service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:29:06.278227 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2059230e-b41e-4bc1-8f6b-0697f9a18add service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:29:06.344373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059230e-b41e-4bc1-8f6b-0697f9a18add service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:29:06.346008 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059230e-b41e-4bc1-8f6b-0697f9a18add service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:29:06.356205 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1322/2619] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:29:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:29:09.121496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34fc05-ea5b-4f45-aa06-2ce4c75f9b85 service swift] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:29:09.131912 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34fc05-ea5b-4f45-aa06-2ce4c75f9b85 service swift] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:29:09.135445 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f2d723-875b-4f19-8200-083102e0dcbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:29:09.136048 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2d723-875b-4f19-8200-083102e0dcbe service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:29:09.136322 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2d723-875b-4f19-8200-083102e0dcbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:29:09.136589 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2d723-875b-4f19-8200-083102e0dcbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:29:09.138451 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f0f2d723-875b-4f19-8200-083102e0dcbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:29:09.152527 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f2d723-875b-4f19-8200-083102e0dcbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:29:09.153943 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f2d723-875b-4f19-8200-083102e0dcbe service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:29:09.163598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1298/2620] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:29:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:29:12.245506 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059230e-b41e-4bc1-8f6b-0697f9a18add service nova] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:29:12.246978 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2059230e-b41e-4bc1-8f6b-0697f9a18add service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:29:12.324255 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059230e-b41e-4bc1-8f6b-0697f9a18add service nova] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:29:12.330235 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f1d4e7-ba99-49aa-8840-5dfdf98153f5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c08fba2d7de42218400c5e652cc3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:29:12.330875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1d4e7-ba99-49aa-8840-5dfdf98153f5 service placement] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:29:12.331159 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1d4e7-ba99-49aa-8840-5dfdf98153f5 service placement] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:29:12.331554 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1d4e7-ba99-49aa-8840-5dfdf98153f5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:29:12.333760 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a2f1d4e7-ba99-49aa-8840-5dfdf98153f5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:29:12.346498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f1d4e7-ba99-49aa-8840-5dfdf98153f5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:29:12.348092 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f1d4e7-ba99-49aa-8840-5dfdf98153f5 service placement] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:29:12.358968 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1323/2621] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:29:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:29:13.051881 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f2d723-875b-4f19-8200-083102e0dcbe service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:29:13.060884 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f2d723-875b-4f19-8200-083102e0dcbe service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:29:13.064901 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea2390-7ebf-41af-b534-c041a544ecae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:29:13.065515 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea2390-7ebf-41af-b534-c041a544ecae service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:29:13.065881 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea2390-7ebf-41af-b534-c041a544ecae service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:29:13.066091 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea2390-7ebf-41af-b534-c041a544ecae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:29:13.067872 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-83ea2390-7ebf-41af-b534-c041a544ecae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:29:13.081468 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ea2390-7ebf-41af-b534-c041a544ecae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:29:13.082932 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ea2390-7ebf-41af-b534-c041a544ecae service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:29:13.094698 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1299/2622] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:29:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:29:16.299544 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f1d4e7-ba99-49aa-8840-5dfdf98153f5 service placement] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:29:16.301306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a2f1d4e7-ba99-49aa-8840-5dfdf98153f5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:29:16.331530 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f1d4e7-ba99-49aa-8840-5dfdf98153f5 service placement] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:29:16.343988 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7f937-596c-48ef-ae9d-dbf0731bf4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:29:16.345429 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7f937-596c-48ef-ae9d-dbf0731bf4fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:29:16.346323 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7f937-596c-48ef-ae9d-dbf0731bf4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:29:16.346873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7f937-596c-48ef-ae9d-dbf0731bf4fc admin admin] PATH_INFO: `/v3/users/5a7f399240ee4003b0e72fd34d96f47b` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:29:16.348441 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b7f937-596c-48ef-ae9d-dbf0731bf4fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a7f399240ee4003b0e72fd34d96f47b)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:29:16.351742 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b7f937-596c-48ef-ae9d-dbf0731bf4fc admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:29:16.428878 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-42b7f937-596c-48ef-ae9d-dbf0731bf4fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a7f399240ee4003b0e72fd34d96f47b'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:29:16.440065 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-42b7f937-596c-48ef-ae9d-dbf0731bf4fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a7f399240ee4003b0e72fd34d96f47b'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:29:16.441073 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-42b7f937-596c-48ef-ae9d-dbf0731bf4fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a7f399240ee4003b0e72fd34d96f47b'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:29:16.502524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-42b7f937-596c-48ef-ae9d-dbf0731bf4fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a7f399240ee4003b0e72fd34d96f47b'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:29:16.519887 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-42b7f937-596c-48ef-ae9d-dbf0731bf4fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a7f399240ee4003b0e72fd34d96f47b'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:29:16.546436 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:29:16.566290 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1324/2623] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:29:16 2022] DELETE /identity/v3/users/5a7f399240ee4003b0e72fd34d96f47b => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:29:17.158734 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea2390-7ebf-41af-b534-c041a544ecae service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:29:17.172402 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea2390-7ebf-41af-b534-c041a544ecae service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:29:17.244992 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292ada7-0920-4b4b-91d9-bec1760baf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:29:17.245918 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292ada7-0920-4b4b-91d9-bec1760baf9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:29:17.246466 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292ada7-0920-4b4b-91d9-bec1760baf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:29:17.247017 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292ada7-0920-4b4b-91d9-bec1760baf9a admin admin] PATH_INFO: `/v3/projects/992437dfa6ef4c678ea5d4f92155cf17` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:29:17.250766 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d292ada7-0920-4b4b-91d9-bec1760baf9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=992437dfa6ef4c678ea5d4f92155cf17)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:29:17.252900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d292ada7-0920-4b4b-91d9-bec1760baf9a admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:29:17.343206 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d292ada7-0920-4b4b-91d9-bec1760baf9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '992437dfa6ef4c678ea5d4f92155cf17', 'request_id': 'req-d292ada7-0920-4b4b-91d9-bec1760baf9a'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:29:17.356508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d292ada7-0920-4b4b-91d9-bec1760baf9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '992437dfa6ef4c678ea5d4f92155cf17', 'request_id': 'req-d292ada7-0920-4b4b-91d9-bec1760baf9a'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:29:17.365939 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d292ada7-0920-4b4b-91d9-bec1760baf9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '992437dfa6ef4c678ea5d4f92155cf17', 'request_id': 'req-d292ada7-0920-4b4b-91d9-bec1760baf9a'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:29:17.394475 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d292ada7-0920-4b4b-91d9-bec1760baf9a admin admin] The token cache is being invalidate because project 992437dfa6ef4c678ea5d4f92155cf17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:29:17.394871 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d292ada7-0920-4b4b-91d9-bec1760baf9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:29:17.397591 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-d292ada7-0920-4b4b-91d9-bec1760baf9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:29:17.402583 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1300/2624] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:29:17 2022] DELETE /identity/v3/projects/992437dfa6ef4c678ea5d4f92155cf17 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:29:17.405863 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7f937-596c-48ef-ae9d-dbf0731bf4fc admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:29:17.408580 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-42b7f937-596c-48ef-ae9d-dbf0731bf4fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:29:17.423292 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7f937-596c-48ef-ae9d-dbf0731bf4fc admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:29:17.500289 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477870db-3680-4baf-8a99-533b53479e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:29:17.500968 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477870db-3680-4baf-8a99-533b53479e83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:29:17.501259 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477870db-3680-4baf-8a99-533b53479e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:29:17.501540 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477870db-3680-4baf-8a99-533b53479e83 admin admin] PATH_INFO: `/v3/users/a55596588f0f4bef9f953bd1ec3c4a87` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:29:17.502775 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477870db-3680-4baf-8a99-533b53479e83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a55596588f0f4bef9f953bd1ec3c4a87)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:29:17.505578 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477870db-3680-4baf-8a99-533b53479e83 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:29:17.567938 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-477870db-3680-4baf-8a99-533b53479e83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a55596588f0f4bef9f953bd1ec3c4a87'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:29:17.568745 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-477870db-3680-4baf-8a99-533b53479e83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a55596588f0f4bef9f953bd1ec3c4a87'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:29:17.569647 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-477870db-3680-4baf-8a99-533b53479e83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a55596588f0f4bef9f953bd1ec3c4a87'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:29:17.594921 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-477870db-3680-4baf-8a99-533b53479e83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a55596588f0f4bef9f953bd1ec3c4a87'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:29:17.613453 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-477870db-3680-4baf-8a99-533b53479e83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a55596588f0f4bef9f953bd1ec3c4a87'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:29:17.637907 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1325/2625] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:29:17 2022] DELETE /identity/v3/users/a55596588f0f4bef9f953bd1ec3c4a87 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:29:18.051684 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292ada7-0920-4b4b-91d9-bec1760baf9a admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:29:18.053399 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d292ada7-0920-4b4b-91d9-bec1760baf9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:29:18.077135 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292ada7-0920-4b4b-91d9-bec1760baf9a admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:29:18.193312 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48b493-c9c4-4850-a19c-4d55c85bfeaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:29:18.193967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48b493-c9c4-4850-a19c-4d55c85bfeaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:29:18.194242 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48b493-c9c4-4850-a19c-4d55c85bfeaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:29:18.194653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48b493-c9c4-4850-a19c-4d55c85bfeaa admin admin] PATH_INFO: `/v3/projects/a0aadeb9f2a64ce6a1cfb9fd546ac7eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:29:18.196162 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48b493-c9c4-4850-a19c-4d55c85bfeaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0aadeb9f2a64ce6a1cfb9fd546ac7eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:29:18.198552 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48b493-c9c4-4850-a19c-4d55c85bfeaa admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:29:18.206858 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:29:18.259159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-7a48b493-c9c4-4850-a19c-4d55c85bfeaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0aadeb9f2a64ce6a1cfb9fd546ac7eb', 'request_id': 'req-7a48b493-c9c4-4850-a19c-4d55c85bfeaa'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:29:18.266565 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-7a48b493-c9c4-4850-a19c-4d55c85bfeaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0aadeb9f2a64ce6a1cfb9fd546ac7eb', 'request_id': 'req-7a48b493-c9c4-4850-a19c-4d55c85bfeaa'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:29:18.273051 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-7a48b493-c9c4-4850-a19c-4d55c85bfeaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0aadeb9f2a64ce6a1cfb9fd546ac7eb', 'request_id': 'req-7a48b493-c9c4-4850-a19c-4d55c85bfeaa'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:29:18.286479 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-7a48b493-c9c4-4850-a19c-4d55c85bfeaa admin admin] The token cache is being invalidate because project a0aadeb9f2a64ce6a1cfb9fd546ac7eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:29:18.286791 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-7a48b493-c9c4-4850-a19c-4d55c85bfeaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:29:18.287496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-7a48b493-c9c4-4850-a19c-4d55c85bfeaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:29:18.289592 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1301/2626] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:29:18 2022] DELETE /identity/v3/projects/a0aadeb9f2a64ce6a1cfb9fd546ac7eb => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:29:18.400001 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477870db-3680-4baf-8a99-533b53479e83 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:29:18.401666 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-477870db-3680-4baf-8a99-533b53479e83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:29:18.422675 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477870db-3680-4baf-8a99-533b53479e83 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:29:18.504150 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4513f1b4-d807-4a59-af5b-016b58a3e411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:29:18.504842 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513f1b4-d807-4a59-af5b-016b58a3e411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:29:18.505163 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513f1b4-d807-4a59-af5b-016b58a3e411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:29:18.505447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513f1b4-d807-4a59-af5b-016b58a3e411 admin admin] PATH_INFO: `/v3/users/0eaa342a6bc84529a5d252c7a9a161ba` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:29:18.506740 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4513f1b4-d807-4a59-af5b-016b58a3e411 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0eaa342a6bc84529a5d252c7a9a161ba)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:29:18.508755 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4513f1b4-d807-4a59-af5b-016b58a3e411 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:29:18.534181 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:29:18.562488 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-4513f1b4-d807-4a59-af5b-016b58a3e411 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0eaa342a6bc84529a5d252c7a9a161ba'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:29:18.563276 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-4513f1b4-d807-4a59-af5b-016b58a3e411 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0eaa342a6bc84529a5d252c7a9a161ba'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:29:18.563916 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-4513f1b4-d807-4a59-af5b-016b58a3e411 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0eaa342a6bc84529a5d252c7a9a161ba'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:29:18.577574 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-4513f1b4-d807-4a59-af5b-016b58a3e411 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0eaa342a6bc84529a5d252c7a9a161ba'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:29:18.584258 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-4513f1b4-d807-4a59-af5b-016b58a3e411 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0eaa342a6bc84529a5d252c7a9a161ba'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:29:18.597195 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1326/2627] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:29:18 2022] DELETE /identity/v3/users/0eaa342a6bc84529a5d252c7a9a161ba => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:29:18.768040 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48b493-c9c4-4850-a19c-4d55c85bfeaa admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:29:18.769564 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-7a48b493-c9c4-4850-a19c-4d55c85bfeaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:29:18.785218 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48b493-c9c4-4850-a19c-4d55c85bfeaa admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:29:18.852401 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f1d11-087f-4bed-a032-c04089addd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:29:18.853087 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f1d11-087f-4bed-a032-c04089addd15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:29:18.853488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f1d11-087f-4bed-a032-c04089addd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:29:18.853889 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f1d11-087f-4bed-a032-c04089addd15 admin admin] PATH_INFO: `/v3/projects/243e75013f5f49e88df72564ac0d2fe7` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:29:18.855444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f1d11-087f-4bed-a032-c04089addd15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=243e75013f5f49e88df72564ac0d2fe7)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:29:18.857469 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f1d11-087f-4bed-a032-c04089addd15 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:29:18.919232 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-157f1d11-087f-4bed-a032-c04089addd15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '243e75013f5f49e88df72564ac0d2fe7', 'request_id': 'req-157f1d11-087f-4bed-a032-c04089addd15'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:29:18.929142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-157f1d11-087f-4bed-a032-c04089addd15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '243e75013f5f49e88df72564ac0d2fe7', 'request_id': 'req-157f1d11-087f-4bed-a032-c04089addd15'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:29:18.938041 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-157f1d11-087f-4bed-a032-c04089addd15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '243e75013f5f49e88df72564ac0d2fe7', 'request_id': 'req-157f1d11-087f-4bed-a032-c04089addd15'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:29:18.955070 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-157f1d11-087f-4bed-a032-c04089addd15 admin admin] The token cache is being invalidate because project 243e75013f5f49e88df72564ac0d2fe7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:29:18.955442 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-157f1d11-087f-4bed-a032-c04089addd15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:29:18.956144 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-157f1d11-087f-4bed-a032-c04089addd15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:29:18.957718 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1302/2628] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:29:18 2022] DELETE /identity/v3/projects/243e75013f5f49e88df72564ac0d2fe7 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:29:18.962739 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95c704-a73c-40d9-b9a9-160bdb7a2d4b None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:29:18.963098 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95c704-a73c-40d9-b9a9-160bdb7a2d4b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:29:18.963472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95c704-a73c-40d9-b9a9-160bdb7a2d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:29:18.991130 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-af95c704-a73c-40d9-b9a9-160bdb7a2d4b None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:29:18.992277 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-af95c704-a73c-40d9-b9a9-160bdb7a2d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:29:19.061686 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1327/2629] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:29:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:29:19.069781 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbedd0d2-aa02-4950-bf9c-0c1e127939bc None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:29:19.070123 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbedd0d2-aa02-4950-bf9c-0c1e127939bc None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:29:19.070405 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbedd0d2-aa02-4950-bf9c-0c1e127939bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:29:19.090984 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-fbedd0d2-aa02-4950-bf9c-0c1e127939bc None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:29:19.093124 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-fbedd0d2-aa02-4950-bf9c-0c1e127939bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:29:19.101370 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1303/2630] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:29:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:29:19.105655 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af95c704-a73c-40d9-b9a9-160bdb7a2d4b None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:29:19.115664 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af95c704-a73c-40d9-b9a9-160bdb7a2d4b None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:29:19.120255 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7786ab-a42e-452e-8890-3c1c3e6634d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:29:19.120870 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7786ab-a42e-452e-8890-3c1c3e6634d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:29:19.121148 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7786ab-a42e-452e-8890-3c1c3e6634d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:29:19.121423 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7786ab-a42e-452e-8890-3c1c3e6634d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:29:19.122182 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7786ab-a42e-452e-8890-3c1c3e6634d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:29:19.122487 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7786ab-a42e-452e-8890-3c1c3e6634d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:29:19.124778 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7786ab-a42e-452e-8890-3c1c3e6634d9 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:29:19.136666 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1328/2631] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:29:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:29:19.141141 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbedd0d2-aa02-4950-bf9c-0c1e127939bc None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:29:19.152281 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbedd0d2-aa02-4950-bf9c-0c1e127939bc None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:29:19.157581 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36937ef0-ded6-441f-8fef-572c45ff57de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:29:19.158248 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36937ef0-ded6-441f-8fef-572c45ff57de admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:29:19.158583 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36937ef0-ded6-441f-8fef-572c45ff57de admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:29:19.158881 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36937ef0-ded6-441f-8fef-572c45ff57de admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:29:19.159526 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36937ef0-ded6-441f-8fef-572c45ff57de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:29:19.161419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36937ef0-ded6-441f-8fef-572c45ff57de admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:29:19.177324 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:29:19.192045 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1304/2632] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:29:19 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:29:19.199764 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7786ab-a42e-452e-8890-3c1c3e6634d9 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:29:19.210379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7786ab-a42e-452e-8890-3c1c3e6634d9 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:29:19.268377 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40330230-facd-446f-860a-b7052377b685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:29:19.268946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40330230-facd-446f-860a-b7052377b685 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:29:19.269211 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40330230-facd-446f-860a-b7052377b685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:29:19.269486 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40330230-facd-446f-860a-b7052377b685 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:29:19.270197 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40330230-facd-446f-860a-b7052377b685 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:29:19.272117 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40330230-facd-446f-860a-b7052377b685 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:29:19.293051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1329/2633] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:29:19 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:29:19.297550 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36937ef0-ded6-441f-8fef-572c45ff57de admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:29:19.307745 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36937ef0-ded6-441f-8fef-572c45ff57de admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:29:19.313494 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740f46a-0613-4798-bda8-7047420653e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:29:19.314303 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740f46a-0613-4798-bda8-7047420653e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:29:19.314590 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740f46a-0613-4798-bda8-7047420653e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:29:19.314931 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740f46a-0613-4798-bda8-7047420653e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:29:19.315994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7740f46a-0613-4798-bda8-7047420653e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:29:19.317575 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7740f46a-0613-4798-bda8-7047420653e5 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:29:19.328558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1305/2634] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:29:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:29:19.332571 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40330230-facd-446f-860a-b7052377b685 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:29:19.343006 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40330230-facd-446f-860a-b7052377b685 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:29:19.347960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88022727-b2a2-4904-8ad7-8db359b03509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:29:19.348591 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88022727-b2a2-4904-8ad7-8db359b03509 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:29:19.348975 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88022727-b2a2-4904-8ad7-8db359b03509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:29:19.349260 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88022727-b2a2-4904-8ad7-8db359b03509 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:29:19.349964 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88022727-b2a2-4904-8ad7-8db359b03509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:29:19.351621 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88022727-b2a2-4904-8ad7-8db359b03509 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:29:19.363756 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1330/2635] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:29:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:29:19.367887 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740f46a-0613-4798-bda8-7047420653e5 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:29:19.377772 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740f46a-0613-4798-bda8-7047420653e5 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:29:19.382331 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88e1e3-f0a4-4311-8fdd-5c49f535c6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:29:19.383005 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88e1e3-f0a4-4311-8fdd-5c49f535c6e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:29:19.383286 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88e1e3-f0a4-4311-8fdd-5c49f535c6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:29:19.383562 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88e1e3-f0a4-4311-8fdd-5c49f535c6e6 admin admin] PATH_INFO: `/v3/projects/e51f9e52588e4008adf20f443c367b02/users/89aa18ddc71c4951bf15fcbd404912e9/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:29:19.398216 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88e1e3-f0a4-4311-8fdd-5c49f535c6e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e51f9e52588e4008adf20f443c367b02, user_id=89aa18ddc71c4951bf15fcbd404912e9, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:29:19.400842 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88e1e3-f0a4-4311-8fdd-5c49f535c6e6 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:29:19.414413 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1306/2636] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:29:19 2022] PUT /identity/v3/projects/e51f9e52588e4008adf20f443c367b02/users/89aa18ddc71c4951bf15fcbd404912e9/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:29:19.419332 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf360b4-7ba5-49dc-9a6f-5a4c84d8d4d7 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:29:19.419654 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf360b4-7ba5-49dc-9a6f-5a4c84d8d4d7 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:29:19.420039 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf360b4-7ba5-49dc-9a6f-5a4c84d8d4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:29:19.428123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:29:19.440436 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-0cf360b4-7ba5-49dc-9a6f-5a4c84d8d4d7 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:29:19.441631 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-0cf360b4-7ba5-49dc-9a6f-5a4c84d8d4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:29:19.500476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1331/2637] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:29:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:29:19.510834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88e1e3-f0a4-4311-8fdd-5c49f535c6e6 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:29:19.512730 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-eb88e1e3-f0a4-4311-8fdd-5c49f535c6e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:29:19.571613 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88e1e3-f0a4-4311-8fdd-5c49f535c6e6 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:29:19.575412 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2cc8b7-097e-4181-b131-6ad68c557dec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:29:19.576556 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2cc8b7-097e-4181-b131-6ad68c557dec service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:29:19.577056 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2cc8b7-097e-4181-b131-6ad68c557dec service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:29:19.577421 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2cc8b7-097e-4181-b131-6ad68c557dec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:29:19.586294 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2cc8b7-097e-4181-b131-6ad68c557dec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:29:19.587725 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2cc8b7-097e-4181-b131-6ad68c557dec service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:29:19.598002 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1307/2638] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:29:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:29:24.438554 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf360b4-7ba5-49dc-9a6f-5a4c84d8d4d7 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:29:24.440292 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-0cf360b4-7ba5-49dc-9a6f-5a4c84d8d4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:29:24.505545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf360b4-7ba5-49dc-9a6f-5a4c84d8d4d7 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:29:24.509601 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d502a09-978d-4367-bef2-2bebefa5272d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:29:24.510632 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d502a09-978d-4367-bef2-2bebefa5272d service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:29:24.511025 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d502a09-978d-4367-bef2-2bebefa5272d service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:29:24.511413 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d502a09-978d-4367-bef2-2bebefa5272d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:29:24.513444 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-1d502a09-978d-4367-bef2-2bebefa5272d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:29:24.589135 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d502a09-978d-4367-bef2-2bebefa5272d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:29:24.590625 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d502a09-978d-4367-bef2-2bebefa5272d service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:29:24.602952 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1332/2639] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:29:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:29:25.175774 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ed4a6-c447-49a9-a023-b473e8b14b54 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:29:25.176338 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ed4a6-c447-49a9-a023-b473e8b14b54 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:29:25.176338 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ed4a6-c447-49a9-a023-b473e8b14b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:29:25.209526 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-3a7ed4a6-c447-49a9-a023-b473e8b14b54 None None] MFA Rules not processed for user `89aa18ddc71c4951bf15fcbd404912e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:29:25.210914 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3a7ed4a6-c447-49a9-a023-b473e8b14b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:29:25.288676 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1308/2640] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:29:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:29:25.305579 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d502a09-978d-4367-bef2-2bebefa5272d service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:29:25.307390 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-1d502a09-978d-4367-bef2-2bebefa5272d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:29:25.378152 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:29:25.400124 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d502a09-978d-4367-bef2-2bebefa5272d service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:29:25.407733 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47cc306-c606-42f8-a792-970d4d535dae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:29:25.408376 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47cc306-c606-42f8-a792-970d4d535dae service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:29:25.408664 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47cc306-c606-42f8-a792-970d4d535dae service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:29:25.408950 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47cc306-c606-42f8-a792-970d4d535dae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:29:25.418839 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47cc306-c606-42f8-a792-970d4d535dae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:29:25.421046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47cc306-c606-42f8-a792-970d4d535dae service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:29:25.431341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1333/2641] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:29:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:29:26.722599 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4767b8f-bbf1-4177-95ed-14371b305973 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:29:26.723478 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4767b8f-bbf1-4177-95ed-14371b305973 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:29:26.724000 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4767b8f-bbf1-4177-95ed-14371b305973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:29:26.744132 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-b4767b8f-bbf1-4177-95ed-14371b305973 None None] MFA Rules not processed for user `8cbc27f4433e4c48a75b216540a9022f`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:29:26.745452 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b4767b8f-bbf1-4177-95ed-14371b305973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:29:26.798380 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:29:26.810590 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1309/2642] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:29:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:29:27.075416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d2002-0fa1-4ca0-af5c-d2b4fdd8ccf8 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:29:27.076018 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d2002-0fa1-4ca0-af5c-d2b4fdd8ccf8 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:29:27.076198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d2002-0fa1-4ca0-af5c-d2b4fdd8ccf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:29:27.096599 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-cc7d2002-0fa1-4ca0-af5c-d2b4fdd8ccf8 None None] MFA Rules not processed for user `89aa18ddc71c4951bf15fcbd404912e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:29:27.097830 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-cc7d2002-0fa1-4ca0-af5c-d2b4fdd8ccf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:29:27.105060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1334/2643] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:29:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:29:31.849536 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313af8c-7fce-448c-9ba8-2afeea981f16 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:29:31.850145 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313af8c-7fce-448c-9ba8-2afeea981f16 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:29:31.850145 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313af8c-7fce-448c-9ba8-2afeea981f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:29:31.879714 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-d313af8c-7fce-448c-9ba8-2afeea981f16 None None] MFA Rules not processed for user `eea5b08eede4409484dc2f4305b5298a`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:29:31.882437 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-d313af8c-7fce-448c-9ba8-2afeea981f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:29:31.901512 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1310/2644] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:29:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:29:37.090965 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=10 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:29:41.871869 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=4 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:30:38.401071 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7d2002-0fa1-4ca0-af5c-d2b4fdd8ccf8 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:30:38.404267 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-cc7d2002-0fa1-4ca0-af5c-d2b4fdd8ccf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:30:38.577289 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7d2002-0fa1-4ca0-af5c-d2b4fdd8ccf8 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:30:38.583927 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd665c-c777-46a1-9d68-1d8693503ed4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c08fba2d7de42218400c5e652cc3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:30:38.584164 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd665c-c777-46a1-9d68-1d8693503ed4 service placement] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:30:38.584421 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd665c-c777-46a1-9d68-1d8693503ed4 service placement] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:30:38.584691 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd665c-c777-46a1-9d68-1d8693503ed4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:30:38.586500 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2ccd665c-c777-46a1-9d68-1d8693503ed4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:30:38.599736 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd665c-c777-46a1-9d68-1d8693503ed4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:30:38.601338 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd665c-c777-46a1-9d68-1d8693503ed4 service placement] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:30:38.612444 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1335/2645] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:30:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:30:42.554822 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795cc1d4-630f-4bd5-b9e7-4d172eb58bb0 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:30:42.555365 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795cc1d4-630f-4bd5-b9e7-4d172eb58bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:30:42.555365 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795cc1d4-630f-4bd5-b9e7-4d172eb58bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:30:42.577568 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-795cc1d4-630f-4bd5-b9e7-4d172eb58bb0 None None] MFA Rules not processed for user `8cbc27f4433e4c48a75b216540a9022f`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:30:42.579716 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-795cc1d4-630f-4bd5-b9e7-4d172eb58bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:30:42.588012 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1311/2646] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:30:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:30:48.607373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=63 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:30:52.570189 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:31:33.766089 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd665c-c777-46a1-9d68-1d8693503ed4 service placement] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:33.785137 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd665c-c777-46a1-9d68-1d8693503ed4 service placement] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:33.788664 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea81a1-6c8c-458f-9f40-d3429a94bb5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:33.789275 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea81a1-6c8c-458f-9f40-d3429a94bb5c service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:33.789553 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea81a1-6c8c-458f-9f40-d3429a94bb5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:33.789827 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea81a1-6c8c-458f-9f40-d3429a94bb5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:33.791608 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b9ea81a1-6c8c-458f-9f40-d3429a94bb5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:31:33.825528 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ea81a1-6c8c-458f-9f40-d3429a94bb5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:33.826982 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ea81a1-6c8c-458f-9f40-d3429a94bb5c service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:33.847757 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1336/2647] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:31:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:31:40.402197 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795cc1d4-630f-4bd5-b9e7-4d172eb58bb0 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:40.403705 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-795cc1d4-630f-4bd5-b9e7-4d172eb58bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:31:40.427257 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795cc1d4-630f-4bd5-b9e7-4d172eb58bb0 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:40.433030 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d2a1d-7d70-4f6c-a408-d45f0594feea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:40.433700 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d2a1d-7d70-4f6c-a408-d45f0594feea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:40.433994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d2a1d-7d70-4f6c-a408-d45f0594feea admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:40.434270 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d2a1d-7d70-4f6c-a408-d45f0594feea admin admin] PATH_INFO: `/v3/users/eea5b08eede4409484dc2f4305b5298a` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:40.435456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52d2a1d-7d70-4f6c-a408-d45f0594feea admin admin] RBAC: Authorizing `identity:delete_user(user_id=eea5b08eede4409484dc2f4305b5298a)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:40.437439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52d2a1d-7d70-4f6c-a408-d45f0594feea admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:40.447390 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea81a1-6c8c-458f-9f40-d3429a94bb5c service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:40.456473 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea81a1-6c8c-458f-9f40-d3429a94bb5c service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:40.460246 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608938e6-5f7e-41b3-adcf-0c3130ee58b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:40.460813 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608938e6-5f7e-41b3-adcf-0c3130ee58b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:40.461123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608938e6-5f7e-41b3-adcf-0c3130ee58b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:40.461403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608938e6-5f7e-41b3-adcf-0c3130ee58b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:40.463235 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-608938e6-5f7e-41b3-adcf-0c3130ee58b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:31:40.493094 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-b52d2a1d-7d70-4f6c-a408-d45f0594feea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eea5b08eede4409484dc2f4305b5298a'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:40.493733 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-b52d2a1d-7d70-4f6c-a408-d45f0594feea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eea5b08eede4409484dc2f4305b5298a'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:40.502322 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-b52d2a1d-7d70-4f6c-a408-d45f0594feea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eea5b08eede4409484dc2f4305b5298a'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:40.502738 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608938e6-5f7e-41b3-adcf-0c3130ee58b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:40.504358 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608938e6-5f7e-41b3-adcf-0c3130ee58b2 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:40.515774 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-b52d2a1d-7d70-4f6c-a408-d45f0594feea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eea5b08eede4409484dc2f4305b5298a'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:40.522991 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-b52d2a1d-7d70-4f6c-a408-d45f0594feea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eea5b08eede4409484dc2f4305b5298a'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:40.525928 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1337/2648] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:31:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 04:31:40.533750 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1312/2649] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:31:40 2022] DELETE /identity/v3/users/eea5b08eede4409484dc2f4305b5298a => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:31:40.710084 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608938e6-5f7e-41b3-adcf-0c3130ee58b2 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:40.721645 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608938e6-5f7e-41b3-adcf-0c3130ee58b2 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:40.784609 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2ab64-3282-45c0-b5db-5968e9378ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:40.785253 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2ab64-3282-45c0-b5db-5968e9378ab2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:40.785535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2ab64-3282-45c0-b5db-5968e9378ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:40.785824 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2ab64-3282-45c0-b5db-5968e9378ab2 admin admin] PATH_INFO: `/v3/projects/3825c2630a4f47e0b029bd0c90875108` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:40.787001 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e2ab64-3282-45c0-b5db-5968e9378ab2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3825c2630a4f47e0b029bd0c90875108)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:40.788902 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e2ab64-3282-45c0-b5db-5968e9378ab2 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:40.962833 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-56e2ab64-3282-45c0-b5db-5968e9378ab2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3825c2630a4f47e0b029bd0c90875108', 'request_id': 'req-56e2ab64-3282-45c0-b5db-5968e9378ab2'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:40.969516 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-56e2ab64-3282-45c0-b5db-5968e9378ab2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3825c2630a4f47e0b029bd0c90875108', 'request_id': 'req-56e2ab64-3282-45c0-b5db-5968e9378ab2'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:40.976111 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-56e2ab64-3282-45c0-b5db-5968e9378ab2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3825c2630a4f47e0b029bd0c90875108', 'request_id': 'req-56e2ab64-3282-45c0-b5db-5968e9378ab2'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:40.985818 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:31:40.991650 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-56e2ab64-3282-45c0-b5db-5968e9378ab2 admin admin] The token cache is being invalidate because project 3825c2630a4f47e0b029bd0c90875108 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:31:40.992114 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-56e2ab64-3282-45c0-b5db-5968e9378ab2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:40.993055 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-56e2ab64-3282-45c0-b5db-5968e9378ab2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:40.997864 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1338/2650] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:31:40 2022] DELETE /identity/v3/projects/3825c2630a4f47e0b029bd0c90875108 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:31:40.999148 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d2a1d-7d70-4f6c-a408-d45f0594feea admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:41.000561 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-b52d2a1d-7d70-4f6c-a408-d45f0594feea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:31:41.014067 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d2a1d-7d70-4f6c-a408-d45f0594feea admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:41.077964 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2ab64-3282-45c0-b5db-5968e9378ab2 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:41.079499 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-56e2ab64-3282-45c0-b5db-5968e9378ab2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:31:41.090375 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e5bb4-db59-4ed6-8456-2b33767159b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:41.091136 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e5bb4-db59-4ed6-8456-2b33767159b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:41.091427 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e5bb4-db59-4ed6-8456-2b33767159b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:41.092004 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e5bb4-db59-4ed6-8456-2b33767159b6 admin admin] PATH_INFO: `/v3/users/fbff1c3ef2cf437b995e1096c360e894` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:41.093444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e5bb4-db59-4ed6-8456-2b33767159b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbff1c3ef2cf437b995e1096c360e894)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:41.097749 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e5bb4-db59-4ed6-8456-2b33767159b6 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:41.150177 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-153e5bb4-db59-4ed6-8456-2b33767159b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbff1c3ef2cf437b995e1096c360e894'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:41.150927 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-153e5bb4-db59-4ed6-8456-2b33767159b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbff1c3ef2cf437b995e1096c360e894'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:41.154855 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2ab64-3282-45c0-b5db-5968e9378ab2 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:41.158982 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-153e5bb4-db59-4ed6-8456-2b33767159b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbff1c3ef2cf437b995e1096c360e894'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:41.159409 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b8b9b3-6ca9-48b5-b458-e4d7b29f9fc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:41.160049 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8b9b3-6ca9-48b5-b458-e4d7b29f9fc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:41.160324 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8b9b3-6ca9-48b5-b458-e4d7b29f9fc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:41.160602 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8b9b3-6ca9-48b5-b458-e4d7b29f9fc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:41.162433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e1b8b9b3-6ca9-48b5-b458-e4d7b29f9fc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:31:41.171889 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-153e5bb4-db59-4ed6-8456-2b33767159b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbff1c3ef2cf437b995e1096c360e894'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:41.174067 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:31:41.180391 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-153e5bb4-db59-4ed6-8456-2b33767159b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbff1c3ef2cf437b995e1096c360e894'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:41.187429 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1313/2651] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:31:40 2022] DELETE /identity/v3/users/fbff1c3ef2cf437b995e1096c360e894 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:31:41.236550 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b8b9b3-6ca9-48b5-b458-e4d7b29f9fc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:41.238138 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b8b9b3-6ca9-48b5-b458-e4d7b29f9fc9 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:41.250240 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1339/2652] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:31:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 04:31:41.443666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e5bb4-db59-4ed6-8456-2b33767159b6 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:41.445257 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-153e5bb4-db59-4ed6-8456-2b33767159b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:31:41.522397 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e5bb4-db59-4ed6-8456-2b33767159b6 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:41.526896 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb771f4-a40c-416e-9413-a8138addf76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:41.527539 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb771f4-a40c-416e-9413-a8138addf76e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:41.527819 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb771f4-a40c-416e-9413-a8138addf76e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:41.528092 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb771f4-a40c-416e-9413-a8138addf76e admin admin] PATH_INFO: `/v3/projects/f2f7e221014e4094831675d8e385da46` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:41.529245 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb771f4-a40c-416e-9413-a8138addf76e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2f7e221014e4094831675d8e385da46)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:41.531194 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb771f4-a40c-416e-9413-a8138addf76e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:41.582476 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1eb771f4-a40c-416e-9413-a8138addf76e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2f7e221014e4094831675d8e385da46', 'request_id': 'req-1eb771f4-a40c-416e-9413-a8138addf76e'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:41.588723 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1eb771f4-a40c-416e-9413-a8138addf76e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2f7e221014e4094831675d8e385da46', 'request_id': 'req-1eb771f4-a40c-416e-9413-a8138addf76e'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:41.594321 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1eb771f4-a40c-416e-9413-a8138addf76e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2f7e221014e4094831675d8e385da46', 'request_id': 'req-1eb771f4-a40c-416e-9413-a8138addf76e'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:41.608713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1eb771f4-a40c-416e-9413-a8138addf76e admin admin] The token cache is being invalidate because project f2f7e221014e4094831675d8e385da46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:31:41.608985 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1eb771f4-a40c-416e-9413-a8138addf76e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:41.609602 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1eb771f4-a40c-416e-9413-a8138addf76e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:41.622080 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1314/2653] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:31:41 2022] DELETE /identity/v3/projects/f2f7e221014e4094831675d8e385da46 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:31:51.250551 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=59 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:31:51.605452 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:31:51.645401 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b8b9b3-6ca9-48b5-b458-e4d7b29f9fc9 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:51.646513 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e1b8b9b3-6ca9-48b5-b458-e4d7b29f9fc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:31:51.661333 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b8b9b3-6ca9-48b5-b458-e4d7b29f9fc9 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:51.719820 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30678170-7792-4a94-be4c-bb59ec75f0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:51.720637 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30678170-7792-4a94-be4c-bb59ec75f0ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:51.721127 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30678170-7792-4a94-be4c-bb59ec75f0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:51.721454 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30678170-7792-4a94-be4c-bb59ec75f0ee admin admin] PATH_INFO: `/v3/users/8cbc27f4433e4c48a75b216540a9022f` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:51.722649 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30678170-7792-4a94-be4c-bb59ec75f0ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cbc27f4433e4c48a75b216540a9022f)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:51.724562 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30678170-7792-4a94-be4c-bb59ec75f0ee admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:51.770013 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-30678170-7792-4a94-be4c-bb59ec75f0ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cbc27f4433e4c48a75b216540a9022f'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:51.770673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-30678170-7792-4a94-be4c-bb59ec75f0ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cbc27f4433e4c48a75b216540a9022f'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:51.771289 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-30678170-7792-4a94-be4c-bb59ec75f0ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cbc27f4433e4c48a75b216540a9022f'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:51.782848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-30678170-7792-4a94-be4c-bb59ec75f0ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cbc27f4433e4c48a75b216540a9022f'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:51.789682 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-30678170-7792-4a94-be4c-bb59ec75f0ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cbc27f4433e4c48a75b216540a9022f'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:51.802524 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1340/2654] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:31:51 2022] DELETE /identity/v3/users/8cbc27f4433e4c48a75b216540a9022f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:31:51.999061 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb771f4-a40c-416e-9413-a8138addf76e admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:52.000754 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-1eb771f4-a40c-416e-9413-a8138addf76e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:31:52.014692 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb771f4-a40c-416e-9413-a8138addf76e admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:52.081533 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61471b7d-eb45-40b5-a97c-e81affda0865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:52.082301 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61471b7d-eb45-40b5-a97c-e81affda0865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:52.082576 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61471b7d-eb45-40b5-a97c-e81affda0865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:52.082843 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61471b7d-eb45-40b5-a97c-e81affda0865 admin admin] PATH_INFO: `/v3/projects/aa51c6088dfa44df848e339d61bac664` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:52.083978 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61471b7d-eb45-40b5-a97c-e81affda0865 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa51c6088dfa44df848e339d61bac664)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:52.085866 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61471b7d-eb45-40b5-a97c-e81affda0865 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:52.130752 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-61471b7d-eb45-40b5-a97c-e81affda0865 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa51c6088dfa44df848e339d61bac664', 'request_id': 'req-61471b7d-eb45-40b5-a97c-e81affda0865'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:52.136685 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-61471b7d-eb45-40b5-a97c-e81affda0865 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa51c6088dfa44df848e339d61bac664', 'request_id': 'req-61471b7d-eb45-40b5-a97c-e81affda0865'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:52.142587 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-61471b7d-eb45-40b5-a97c-e81affda0865 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa51c6088dfa44df848e339d61bac664', 'request_id': 'req-61471b7d-eb45-40b5-a97c-e81affda0865'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:52.154098 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-61471b7d-eb45-40b5-a97c-e81affda0865 admin admin] The token cache is being invalidate because project aa51c6088dfa44df848e339d61bac664 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:31:52.154382 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-61471b7d-eb45-40b5-a97c-e81affda0865 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:52.154988 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-61471b7d-eb45-40b5-a97c-e81affda0865 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:52.156488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1315/2655] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:31:51 2022] DELETE /identity/v3/projects/aa51c6088dfa44df848e339d61bac664 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:31:52.162228 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b475f0-86c5-4cae-b451-153960ef1e3e None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:52.162518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b475f0-86c5-4cae-b451-153960ef1e3e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:52.162793 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b475f0-86c5-4cae-b451-153960ef1e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:52.225892 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-c8b475f0-86c5-4cae-b451-153960ef1e3e None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:31:52.227452 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c8b475f0-86c5-4cae-b451-153960ef1e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:31:52.271840 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:31:52.288849 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1341/2656] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:31:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:31:52.296730 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1209d24-4f3a-4053-80cf-c83da5993235 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:52.297043 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1209d24-4f3a-4053-80cf-c83da5993235 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:52.297319 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1209d24-4f3a-4053-80cf-c83da5993235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:52.317443 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-a1209d24-4f3a-4053-80cf-c83da5993235 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:31:52.319077 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-a1209d24-4f3a-4053-80cf-c83da5993235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:31:52.329429 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1316/2657] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:31:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:31:52.334491 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b475f0-86c5-4cae-b451-153960ef1e3e None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:52.344102 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b475f0-86c5-4cae-b451-153960ef1e3e None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:52.349164 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8c0a8-b1d9-4274-90a6-9b42db83ebe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:52.349781 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8c0a8-b1d9-4274-90a6-9b42db83ebe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:52.350083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8c0a8-b1d9-4274-90a6-9b42db83ebe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:52.350360 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8c0a8-b1d9-4274-90a6-9b42db83ebe9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:52.351055 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc8c0a8-b1d9-4274-90a6-9b42db83ebe9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:31:52.351356 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc8c0a8-b1d9-4274-90a6-9b42db83ebe9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:52.352917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc8c0a8-b1d9-4274-90a6-9b42db83ebe9 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:52.364288 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1342/2658] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:31:52.368450 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1209d24-4f3a-4053-80cf-c83da5993235 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:52.377603 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1209d24-4f3a-4053-80cf-c83da5993235 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:52.382216 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501e302-aa3e-452e-91ec-c7163ca6c3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:52.382816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501e302-aa3e-452e-91ec-c7163ca6c3cc admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:52.383092 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501e302-aa3e-452e-91ec-c7163ca6c3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:52.383364 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501e302-aa3e-452e-91ec-c7163ca6c3cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:52.384032 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3501e302-aa3e-452e-91ec-c7163ca6c3cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:52.385914 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3501e302-aa3e-452e-91ec-c7163ca6c3cc admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:52.423337 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1317/2659] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:31:52 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:31:52.427586 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8c0a8-b1d9-4274-90a6-9b42db83ebe9 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:52.437200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8c0a8-b1d9-4274-90a6-9b42db83ebe9 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:52.497059 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5097a77-3709-46cd-9390-58e4523efa0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:52.497673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5097a77-3709-46cd-9390-58e4523efa0c admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:52.497949 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5097a77-3709-46cd-9390-58e4523efa0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:52.498250 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5097a77-3709-46cd-9390-58e4523efa0c admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:52.498903 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5097a77-3709-46cd-9390-58e4523efa0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:52.500778 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5097a77-3709-46cd-9390-58e4523efa0c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:52.521734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1343/2660] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:31:52 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:31:52.525952 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501e302-aa3e-452e-91ec-c7163ca6c3cc admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:52.535990 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501e302-aa3e-452e-91ec-c7163ca6c3cc admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:52.540465 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22574350-c988-46f9-af72-1f5fb5a5423c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:52.541069 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22574350-c988-46f9-af72-1f5fb5a5423c admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:52.541347 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22574350-c988-46f9-af72-1f5fb5a5423c admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:52.541616 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22574350-c988-46f9-af72-1f5fb5a5423c admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:52.542413 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22574350-c988-46f9-af72-1f5fb5a5423c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:52.543990 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22574350-c988-46f9-af72-1f5fb5a5423c admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:52.553917 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1318/2661] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:31:52.557920 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5097a77-3709-46cd-9390-58e4523efa0c admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:52.568588 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5097a77-3709-46cd-9390-58e4523efa0c admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:52.573425 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc4296-106a-4715-9d31-f6b4c11ff3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:52.574065 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc4296-106a-4715-9d31-f6b4c11ff3e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:52.574353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc4296-106a-4715-9d31-f6b4c11ff3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:52.574628 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc4296-106a-4715-9d31-f6b4c11ff3e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:52.575327 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecc4296-106a-4715-9d31-f6b4c11ff3e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:52.576901 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecc4296-106a-4715-9d31-f6b4c11ff3e7 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:52.587199 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1344/2662] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:31:52.591099 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22574350-c988-46f9-af72-1f5fb5a5423c admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:52.600915 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22574350-c988-46f9-af72-1f5fb5a5423c admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:52.605398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5372d-77df-46e4-8843-d0e4d23ef5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:52.606083 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5372d-77df-46e4-8843-d0e4d23ef5f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:52.606629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5372d-77df-46e4-8843-d0e4d23ef5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:52.606629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5372d-77df-46e4-8843-d0e4d23ef5f7 admin admin] PATH_INFO: `/v3/projects/88755b9d12304b08ae62ad70d897a91c/users/ddae5752daf346ccb4e17b9420a22ff5/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:52.620795 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba5372d-77df-46e4-8843-d0e4d23ef5f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88755b9d12304b08ae62ad70d897a91c, user_id=ddae5752daf346ccb4e17b9420a22ff5, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:52.623491 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba5372d-77df-46e4-8843-d0e4d23ef5f7 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:52.637247 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1319/2663] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:31:52 2022] PUT /identity/v3/projects/88755b9d12304b08ae62ad70d897a91c/users/ddae5752daf346ccb4e17b9420a22ff5/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:31:52.648786 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe3fe5-9c75-4573-adef-97c042789bf8 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:52.649093 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe3fe5-9c75-4573-adef-97c042789bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:52.649381 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe3fe5-9c75-4573-adef-97c042789bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:52.676735 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-50fe3fe5-9c75-4573-adef-97c042789bf8 None None] MFA Rules not processed for user `ddae5752daf346ccb4e17b9420a22ff5`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:31:52.677947 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-50fe3fe5-9c75-4573-adef-97c042789bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:31:52.740084 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1345/2664] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:31:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:31:52.744570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5372d-77df-46e4-8843-d0e4d23ef5f7 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:52.764083 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:31:52.808510 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5372d-77df-46e4-8843-d0e4d23ef5f7 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:52.813110 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408cb0cf-482c-40ce-bd68-f6595b08ef5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:52.813717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408cb0cf-482c-40ce-bd68-f6595b08ef5e admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:52.814729 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408cb0cf-482c-40ce-bd68-f6595b08ef5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:52.815006 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408cb0cf-482c-40ce-bd68-f6595b08ef5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:52.815672 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408cb0cf-482c-40ce-bd68-f6595b08ef5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:52.817651 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408cb0cf-482c-40ce-bd68-f6595b08ef5e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:52.844276 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1320/2665] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:31:52 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:31:52.848572 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe3fe5-9c75-4573-adef-97c042789bf8 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:52.859156 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe3fe5-9c75-4573-adef-97c042789bf8 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:52.903979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:31:52.920758 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6878e4-f52e-4297-857e-4c5e4ecb9e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:52.921389 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6878e4-f52e-4297-857e-4c5e4ecb9e98 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:52.921673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6878e4-f52e-4297-857e-4c5e4ecb9e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:52.921949 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6878e4-f52e-4297-857e-4c5e4ecb9e98 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:52.922844 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6878e4-f52e-4297-857e-4c5e4ecb9e98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:52.926131 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6878e4-f52e-4297-857e-4c5e4ecb9e98 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:52.951439 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1346/2666] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:31:52 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:31:52.951977 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408cb0cf-482c-40ce-bd68-f6595b08ef5e admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:52.962565 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408cb0cf-482c-40ce-bd68-f6595b08ef5e admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:52.967137 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe4a1d-aff7-47b9-aca7-697f3ed7f026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:52.967746 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe4a1d-aff7-47b9-aca7-697f3ed7f026 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:52.968026 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe4a1d-aff7-47b9-aca7-697f3ed7f026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:52.968300 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe4a1d-aff7-47b9-aca7-697f3ed7f026 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:52.968995 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe4a1d-aff7-47b9-aca7-697f3ed7f026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:52.970869 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe4a1d-aff7-47b9-aca7-697f3ed7f026 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:52.981524 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1321/2667] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:31:52.986347 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6878e4-f52e-4297-857e-4c5e4ecb9e98 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:52.996283 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6878e4-f52e-4297-857e-4c5e4ecb9e98 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:53.000853 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b08e1-bc76-4213-a227-679e76f81595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:53.002189 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b08e1-bc76-4213-a227-679e76f81595 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:53.002476 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b08e1-bc76-4213-a227-679e76f81595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:53.002758 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b08e1-bc76-4213-a227-679e76f81595 admin admin] PATH_INFO: `/v3/projects/dfd57f8986294435b946302c47c4754e/users/176780c325bd4b3bb60927f040feed63/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:53.017026 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b08e1-bc76-4213-a227-679e76f81595 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfd57f8986294435b946302c47c4754e, user_id=176780c325bd4b3bb60927f040feed63, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:53.020000 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b08e1-bc76-4213-a227-679e76f81595 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:53.032855 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1347/2668] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:31:52 2022] PUT /identity/v3/projects/dfd57f8986294435b946302c47c4754e/users/176780c325bd4b3bb60927f040feed63/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:31:53.036786 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe4a1d-aff7-47b9-aca7-697f3ed7f026 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:53.046665 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe4a1d-aff7-47b9-aca7-697f3ed7f026 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:53.102689 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d58b1-c335-4db2-ac51-d24e8ee9a733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:53.103306 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d58b1-c335-4db2-ac51-d24e8ee9a733 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:53.103586 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d58b1-c335-4db2-ac51-d24e8ee9a733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:53.103860 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d58b1-c335-4db2-ac51-d24e8ee9a733 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:53.104577 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53d58b1-c335-4db2-ac51-d24e8ee9a733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:53.106189 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53d58b1-c335-4db2-ac51-d24e8ee9a733 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:53.116398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1322/2669] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:31:53.120570 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b08e1-bc76-4213-a227-679e76f81595 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:53.131661 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b08e1-bc76-4213-a227-679e76f81595 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:53.136415 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c0c6d-471d-478e-8548-c49bd0786acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:53.137083 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c0c6d-471d-478e-8548-c49bd0786acf admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:53.137366 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c0c6d-471d-478e-8548-c49bd0786acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:53.137640 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c0c6d-471d-478e-8548-c49bd0786acf admin admin] PATH_INFO: `/v3/projects/dfd57f8986294435b946302c47c4754e/users/176780c325bd4b3bb60927f040feed63/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:53.139615 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c0c6d-471d-478e-8548-c49bd0786acf admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfd57f8986294435b946302c47c4754e, user_id=176780c325bd4b3bb60927f040feed63, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:53.142364 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c0c6d-471d-478e-8548-c49bd0786acf admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:53.152392 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1348/2670] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:31:53 2022] PUT /identity/v3/projects/dfd57f8986294435b946302c47c4754e/users/176780c325bd4b3bb60927f040feed63/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:31:53.162803 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb2d98-4278-45bf-a9f5-0ecd8fbc3bf8 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:53.163096 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb2d98-4278-45bf-a9f5-0ecd8fbc3bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:53.163374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb2d98-4278-45bf-a9f5-0ecd8fbc3bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:53.191631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-22eb2d98-4278-45bf-a9f5-0ecd8fbc3bf8 None None] MFA Rules not processed for user `176780c325bd4b3bb60927f040feed63`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:31:53.192834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-22eb2d98-4278-45bf-a9f5-0ecd8fbc3bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:31:53.259166 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1323/2671] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:31:53 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:31:53.270870 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c0c6d-471d-478e-8548-c49bd0786acf admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:53.272585 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f61c0c6d-471d-478e-8548-c49bd0786acf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:31:53.351090 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c0c6d-471d-478e-8548-c49bd0786acf admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:53.354767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924351bf-6d54-4bf5-b4b0-25edede6fd81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:53.355385 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924351bf-6d54-4bf5-b4b0-25edede6fd81 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:53.355668 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924351bf-6d54-4bf5-b4b0-25edede6fd81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:53.355944 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924351bf-6d54-4bf5-b4b0-25edede6fd81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:53.365509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924351bf-6d54-4bf5-b4b0-25edede6fd81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:53.366956 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924351bf-6d54-4bf5-b4b0-25edede6fd81 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:53.376222 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1349/2672] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:31:53 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:31:53.756326 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb2d98-4278-45bf-a9f5-0ecd8fbc3bf8 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:53.769424 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:31:53.819472 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb2d98-4278-45bf-a9f5-0ecd8fbc3bf8 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:53.824211 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd14570-464e-4df7-bc90-16de242077ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:53.824843 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd14570-464e-4df7-bc90-16de242077ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:53.825126 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd14570-464e-4df7-bc90-16de242077ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:53.825400 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd14570-464e-4df7-bc90-16de242077ed admin admin] PATH_INFO: `/v3/users/ddae5752daf346ccb4e17b9420a22ff5` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:53.826714 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd14570-464e-4df7-bc90-16de242077ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddae5752daf346ccb4e17b9420a22ff5)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:53.828825 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd14570-464e-4df7-bc90-16de242077ed admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:53.872733 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1cd14570-464e-4df7-bc90-16de242077ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddae5752daf346ccb4e17b9420a22ff5'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:53.873385 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1cd14570-464e-4df7-bc90-16de242077ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddae5752daf346ccb4e17b9420a22ff5'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:53.881808 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1cd14570-464e-4df7-bc90-16de242077ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddae5752daf346ccb4e17b9420a22ff5'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:53.893993 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1cd14570-464e-4df7-bc90-16de242077ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddae5752daf346ccb4e17b9420a22ff5'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:53.901191 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-1cd14570-464e-4df7-bc90-16de242077ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddae5752daf346ccb4e17b9420a22ff5'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:53.907981 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1324/2673] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:31:53 2022] DELETE /identity/v3/users/ddae5752daf346ccb4e17b9420a22ff5 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:31:53.913938 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18967cc0-5d20-4d8b-8af4-323ab378995b None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:53.914269 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18967cc0-5d20-4d8b-8af4-323ab378995b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:53.914552 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18967cc0-5d20-4d8b-8af4-323ab378995b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:53.933802 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-18967cc0-5d20-4d8b-8af4-323ab378995b None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:31:53.935021 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-18967cc0-5d20-4d8b-8af4-323ab378995b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:31:53.993221 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1350/2674] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:31:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:31:54.005386 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd14570-464e-4df7-bc90-16de242077ed admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:54.006819 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-1cd14570-464e-4df7-bc90-16de242077ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:31:54.063978 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd14570-464e-4df7-bc90-16de242077ed admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:54.067992 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd50b4-535f-4c48-ac86-332f3bdc96e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:54.068607 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd50b4-535f-4c48-ac86-332f3bdc96e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:54.068888 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd50b4-535f-4c48-ac86-332f3bdc96e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:54.069163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd50b4-535f-4c48-ac86-332f3bdc96e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:54.078340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cd50b4-535f-4c48-ac86-332f3bdc96e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:54.080258 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cd50b4-535f-4c48-ac86-332f3bdc96e8 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:54.092481 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1325/2675] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:31:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:31:54.367608 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18967cc0-5d20-4d8b-8af4-323ab378995b None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:54.369107 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-18967cc0-5d20-4d8b-8af4-323ab378995b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:31:54.383231 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18967cc0-5d20-4d8b-8af4-323ab378995b None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:54.387967 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840c94b-18a7-4c56-8340-97d84f9accba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:54.388699 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840c94b-18a7-4c56-8340-97d84f9accba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:54.389048 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840c94b-18a7-4c56-8340-97d84f9accba admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:54.389385 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840c94b-18a7-4c56-8340-97d84f9accba admin admin] PATH_INFO: `/v3/projects/88755b9d12304b08ae62ad70d897a91c` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:54.390807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840c94b-18a7-4c56-8340-97d84f9accba admin admin] RBAC: Authorizing `identity:delete_project(project_id=88755b9d12304b08ae62ad70d897a91c)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:54.393236 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840c94b-18a7-4c56-8340-97d84f9accba admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:54.410867 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:31:54.441888 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0840c94b-18a7-4c56-8340-97d84f9accba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88755b9d12304b08ae62ad70d897a91c', 'request_id': 'req-0840c94b-18a7-4c56-8340-97d84f9accba'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:54.448265 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0840c94b-18a7-4c56-8340-97d84f9accba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88755b9d12304b08ae62ad70d897a91c', 'request_id': 'req-0840c94b-18a7-4c56-8340-97d84f9accba'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:54.454618 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0840c94b-18a7-4c56-8340-97d84f9accba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88755b9d12304b08ae62ad70d897a91c', 'request_id': 'req-0840c94b-18a7-4c56-8340-97d84f9accba'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:54.467954 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0840c94b-18a7-4c56-8340-97d84f9accba admin admin] The token cache is being invalidate because project 88755b9d12304b08ae62ad70d897a91c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:31:54.468243 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0840c94b-18a7-4c56-8340-97d84f9accba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:54.468868 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0840c94b-18a7-4c56-8340-97d84f9accba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:54.470408 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1351/2676] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:31:54 2022] DELETE /identity/v3/projects/88755b9d12304b08ae62ad70d897a91c => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:31:54.474935 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd50b4-535f-4c48-ac86-332f3bdc96e8 service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:54.484439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd50b4-535f-4c48-ac86-332f3bdc96e8 service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:54.517599 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:31:54.544686 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c24d479-8a53-4f72-9313-79b12d0de8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:54.545330 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24d479-8a53-4f72-9313-79b12d0de8f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:54.545609 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24d479-8a53-4f72-9313-79b12d0de8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:54.545883 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24d479-8a53-4f72-9313-79b12d0de8f3 admin admin] PATH_INFO: `/v3/users/176780c325bd4b3bb60927f040feed63` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:54.547088 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c24d479-8a53-4f72-9313-79b12d0de8f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=176780c325bd4b3bb60927f040feed63)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:54.548998 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c24d479-8a53-4f72-9313-79b12d0de8f3 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:54.592779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4c24d479-8a53-4f72-9313-79b12d0de8f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '176780c325bd4b3bb60927f040feed63'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:54.593419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4c24d479-8a53-4f72-9313-79b12d0de8f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '176780c325bd4b3bb60927f040feed63'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:54.600976 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4c24d479-8a53-4f72-9313-79b12d0de8f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '176780c325bd4b3bb60927f040feed63'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:54.612899 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4c24d479-8a53-4f72-9313-79b12d0de8f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '176780c325bd4b3bb60927f040feed63'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:54.619858 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-4c24d479-8a53-4f72-9313-79b12d0de8f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '176780c325bd4b3bb60927f040feed63'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:54.625662 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1326/2677] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:31:54 2022] DELETE /identity/v3/users/176780c325bd4b3bb60927f040feed63 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:31:54.904888 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840c94b-18a7-4c56-8340-97d84f9accba admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:54.906485 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-0840c94b-18a7-4c56-8340-97d84f9accba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:31:54.920410 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840c94b-18a7-4c56-8340-97d84f9accba admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:54.979583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9423052a-256e-44ac-8451-06623d04160c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:54.980247 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423052a-256e-44ac-8451-06623d04160c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:54.980525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423052a-256e-44ac-8451-06623d04160c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:54.980797 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423052a-256e-44ac-8451-06623d04160c admin admin] PATH_INFO: `/v3/projects/dfd57f8986294435b946302c47c4754e` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:54.981937 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9423052a-256e-44ac-8451-06623d04160c admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfd57f8986294435b946302c47c4754e)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:54.983849 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9423052a-256e-44ac-8451-06623d04160c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:55.031573 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9423052a-256e-44ac-8451-06623d04160c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfd57f8986294435b946302c47c4754e', 'request_id': 'req-9423052a-256e-44ac-8451-06623d04160c'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:55.038955 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9423052a-256e-44ac-8451-06623d04160c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfd57f8986294435b946302c47c4754e', 'request_id': 'req-9423052a-256e-44ac-8451-06623d04160c'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:55.044322 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9423052a-256e-44ac-8451-06623d04160c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfd57f8986294435b946302c47c4754e', 'request_id': 'req-9423052a-256e-44ac-8451-06623d04160c'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:55.058292 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9423052a-256e-44ac-8451-06623d04160c admin admin] The token cache is being invalidate because project dfd57f8986294435b946302c47c4754e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:31:55.058690 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9423052a-256e-44ac-8451-06623d04160c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:55.059314 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9423052a-256e-44ac-8451-06623d04160c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:31:55.061252 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1352/2678] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:31:54 2022] DELETE /identity/v3/projects/dfd57f8986294435b946302c47c4754e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:31:55.067603 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47420f28-546a-492e-be7f-4f27b9551942 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:55.068062 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47420f28-546a-492e-be7f-4f27b9551942 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:55.068158 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47420f28-546a-492e-be7f-4f27b9551942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:55.087706 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-47420f28-546a-492e-be7f-4f27b9551942 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:31:55.088897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-47420f28-546a-492e-be7f-4f27b9551942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:31:55.147496 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1327/2679] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:31:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:31:55.155207 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4020cb9d-bfa5-4314-b6b0-c790c54566c5 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:55.155499 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4020cb9d-bfa5-4314-b6b0-c790c54566c5 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:55.155782 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4020cb9d-bfa5-4314-b6b0-c790c54566c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:55.176661 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-4020cb9d-bfa5-4314-b6b0-c790c54566c5 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:31:55.177860 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-4020cb9d-bfa5-4314-b6b0-c790c54566c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:31:55.185704 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1353/2680] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:31:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:31:55.189846 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47420f28-546a-492e-be7f-4f27b9551942 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:55.200796 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47420f28-546a-492e-be7f-4f27b9551942 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:55.205530 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed31798-dbd7-416e-9145-d78dbbf4d85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:55.206291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed31798-dbd7-416e-9145-d78dbbf4d85e admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:55.206561 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed31798-dbd7-416e-9145-d78dbbf4d85e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:55.206869 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed31798-dbd7-416e-9145-d78dbbf4d85e admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:55.207638 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed31798-dbd7-416e-9145-d78dbbf4d85e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:31:55.207967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed31798-dbd7-416e-9145-d78dbbf4d85e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:55.209613 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed31798-dbd7-416e-9145-d78dbbf4d85e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:55.221115 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1328/2681] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:31:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:31:55.225626 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4020cb9d-bfa5-4314-b6b0-c790c54566c5 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:55.234695 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4020cb9d-bfa5-4314-b6b0-c790c54566c5 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:55.239731 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a441455-c35e-4c0c-b158-9cefa53a2f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:55.240327 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a441455-c35e-4c0c-b158-9cefa53a2f3e admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:55.240601 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a441455-c35e-4c0c-b158-9cefa53a2f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:55.240868 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a441455-c35e-4c0c-b158-9cefa53a2f3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:55.241539 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a441455-c35e-4c0c-b158-9cefa53a2f3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:55.243424 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a441455-c35e-4c0c-b158-9cefa53a2f3e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:55.271783 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1354/2682] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:31:55 2022] POST /identity/v3/projects => generated 365 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:31:55.273879 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed31798-dbd7-416e-9145-d78dbbf4d85e admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:55.283703 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed31798-dbd7-416e-9145-d78dbbf4d85e admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:55.324354 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:31:55.342837 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76409d9a-0345-46ee-a41b-d3cd0da83ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:55.343437 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76409d9a-0345-46ee-a41b-d3cd0da83ba0 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:55.343715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76409d9a-0345-46ee-a41b-d3cd0da83ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:55.343983 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76409d9a-0345-46ee-a41b-d3cd0da83ba0 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:55.344644 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76409d9a-0345-46ee-a41b-d3cd0da83ba0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:55.346634 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76409d9a-0345-46ee-a41b-d3cd0da83ba0 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:55.373155 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1329/2683] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:31:55 2022] POST /identity/v3/users => generated 283 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:31:55.377497 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a441455-c35e-4c0c-b158-9cefa53a2f3e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:55.384763 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:31:55.388999 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a441455-c35e-4c0c-b158-9cefa53a2f3e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:55.395126 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de7ba7-b3d5-4426-9410-e38451273be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:55.395972 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de7ba7-b3d5-4426-9410-e38451273be2 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:55.396420 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de7ba7-b3d5-4426-9410-e38451273be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:55.396706 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de7ba7-b3d5-4426-9410-e38451273be2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:55.397418 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42de7ba7-b3d5-4426-9410-e38451273be2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:55.399095 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42de7ba7-b3d5-4426-9410-e38451273be2 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:55.409746 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1355/2684] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:31:55.413868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76409d9a-0345-46ee-a41b-d3cd0da83ba0 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:55.425196 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76409d9a-0345-46ee-a41b-d3cd0da83ba0 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:55.430561 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a5fea2-3d3f-4578-8c78-9e01c4ca1091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:55.431322 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5fea2-3d3f-4578-8c78-9e01c4ca1091 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:55.431611 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5fea2-3d3f-4578-8c78-9e01c4ca1091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:55.431986 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5fea2-3d3f-4578-8c78-9e01c4ca1091 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:55.432864 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a5fea2-3d3f-4578-8c78-9e01c4ca1091 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:55.434756 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a5fea2-3d3f-4578-8c78-9e01c4ca1091 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:55.445713 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1330/2685] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:31:55.450195 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de7ba7-b3d5-4426-9410-e38451273be2 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:55.460076 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de7ba7-b3d5-4426-9410-e38451273be2 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:55.468574 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ac6b6-b69a-4351-adbc-292147b88a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:55.469278 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ac6b6-b69a-4351-adbc-292147b88a9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:55.469561 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ac6b6-b69a-4351-adbc-292147b88a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:55.469873 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ac6b6-b69a-4351-adbc-292147b88a9c admin admin] PATH_INFO: `/v3/projects/aa498b2b2c0a4b72a9f0e70b4a3e8442/users/74a14d5ed67c450599b447c3b08d08ae/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:55.485396 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334ac6b6-b69a-4351-adbc-292147b88a9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa498b2b2c0a4b72a9f0e70b4a3e8442, user_id=74a14d5ed67c450599b447c3b08d08ae, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:55.488803 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334ac6b6-b69a-4351-adbc-292147b88a9c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:55.502120 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1356/2686] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:31:55 2022] PUT /identity/v3/projects/aa498b2b2c0a4b72a9f0e70b4a3e8442/users/74a14d5ed67c450599b447c3b08d08ae/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:31:55.569469 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee31425-2300-4d73-90f0-aef7438b4643 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:55.569865 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee31425-2300-4d73-90f0-aef7438b4643 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:55.570239 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee31425-2300-4d73-90f0-aef7438b4643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:55.598144 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-dee31425-2300-4d73-90f0-aef7438b4643 None None] MFA Rules not processed for user `74a14d5ed67c450599b447c3b08d08ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:31:55.599375 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-dee31425-2300-4d73-90f0-aef7438b4643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:31:55.668912 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1331/2687] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:31:55 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:31:55.673392 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ac6b6-b69a-4351-adbc-292147b88a9c admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:55.734838 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ac6b6-b69a-4351-adbc-292147b88a9c admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:55.740403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a1e2d-105d-4800-b7ff-50a49fab88d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:55.741033 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a1e2d-105d-4800-b7ff-50a49fab88d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:55.741317 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a1e2d-105d-4800-b7ff-50a49fab88d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:55.741594 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a1e2d-105d-4800-b7ff-50a49fab88d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:55.742317 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a1e2d-105d-4800-b7ff-50a49fab88d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:55.744297 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a1e2d-105d-4800-b7ff-50a49fab88d5 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:55.773221 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1357/2688] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:31:55 2022] POST /identity/v3/projects => generated 363 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:31:55.777738 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee31425-2300-4d73-90f0-aef7438b4643 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:55.789115 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee31425-2300-4d73-90f0-aef7438b4643 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:55.845485 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a8e01-27fb-4a3b-be89-02af4af2f9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:55.846385 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a8e01-27fb-4a3b-be89-02af4af2f9ee admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:55.846702 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a8e01-27fb-4a3b-be89-02af4af2f9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:55.846983 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a8e01-27fb-4a3b-be89-02af4af2f9ee admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:55.847666 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a8e01-27fb-4a3b-be89-02af4af2f9ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:55.849590 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a8e01-27fb-4a3b-be89-02af4af2f9ee admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:55.874441 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1332/2689] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:31:55 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:31:55.875472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a1e2d-105d-4800-b7ff-50a49fab88d5 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:55.886094 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a1e2d-105d-4800-b7ff-50a49fab88d5 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:55.890746 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b5731-f55a-4315-9fa1-15ddf6d94d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:55.891359 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b5731-f55a-4315-9fa1-15ddf6d94d01 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:55.891639 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b5731-f55a-4315-9fa1-15ddf6d94d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:55.891912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b5731-f55a-4315-9fa1-15ddf6d94d01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:55.892609 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0b5731-f55a-4315-9fa1-15ddf6d94d01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:55.894802 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0b5731-f55a-4315-9fa1-15ddf6d94d01 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:55.908946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1358/2690] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 04:31:55.910244 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a8e01-27fb-4a3b-be89-02af4af2f9ee admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:55.921053 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a8e01-27fb-4a3b-be89-02af4af2f9ee admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:55.926407 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b99bf-d5b0-4169-b647-750ac1cae86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:55.927234 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b99bf-d5b0-4169-b647-750ac1cae86f admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:55.927838 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b99bf-d5b0-4169-b647-750ac1cae86f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:55.927960 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b99bf-d5b0-4169-b647-750ac1cae86f admin admin] PATH_INFO: `/v3/projects/f7d9454ca1b94105bed533995bda4a14/users/944700c14fb741e2b3370e777adb76e2/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:55.943420 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8b99bf-d5b0-4169-b647-750ac1cae86f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7d9454ca1b94105bed533995bda4a14, user_id=944700c14fb741e2b3370e777adb76e2, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:55.946225 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8b99bf-d5b0-4169-b647-750ac1cae86f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:55.961100 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1333/2691] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:31:55 2022] PUT /identity/v3/projects/f7d9454ca1b94105bed533995bda4a14/users/944700c14fb741e2b3370e777adb76e2/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:31:55.965187 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b5731-f55a-4315-9fa1-15ddf6d94d01 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:55.975904 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b5731-f55a-4315-9fa1-15ddf6d94d01 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:56.039079 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75647536-86fc-45c5-9976-dc4c5ba63067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:56.039319 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75647536-86fc-45c5-9976-dc4c5ba63067 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:56.039593 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75647536-86fc-45c5-9976-dc4c5ba63067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:56.039867 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75647536-86fc-45c5-9976-dc4c5ba63067 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:56.040583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75647536-86fc-45c5-9976-dc4c5ba63067 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:56.042193 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75647536-86fc-45c5-9976-dc4c5ba63067 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:56.052509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1359/2692] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 04:31:56.056563 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b99bf-d5b0-4169-b647-750ac1cae86f admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:56.067129 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b99bf-d5b0-4169-b647-750ac1cae86f admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:56.071957 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff51a73-3a61-4e8b-aa1d-cb3f8c432bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:56.072627 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff51a73-3a61-4e8b-aa1d-cb3f8c432bce admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:56.073180 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff51a73-3a61-4e8b-aa1d-cb3f8c432bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:56.073445 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff51a73-3a61-4e8b-aa1d-cb3f8c432bce admin admin] PATH_INFO: `/v3/projects/f7d9454ca1b94105bed533995bda4a14/users/944700c14fb741e2b3370e777adb76e2/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:56.075409 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff51a73-3a61-4e8b-aa1d-cb3f8c432bce admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7d9454ca1b94105bed533995bda4a14, user_id=944700c14fb741e2b3370e777adb76e2, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:56.078091 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff51a73-3a61-4e8b-aa1d-cb3f8c432bce admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:56.088948 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1334/2693] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:31:56 2022] PUT /identity/v3/projects/f7d9454ca1b94105bed533995bda4a14/users/944700c14fb741e2b3370e777adb76e2/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:31:56.099532 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57572f93-6d88-4d96-94f5-47214ba0da65 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:56.099825 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57572f93-6d88-4d96-94f5-47214ba0da65 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:56.100113 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57572f93-6d88-4d96-94f5-47214ba0da65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:56.127616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-57572f93-6d88-4d96-94f5-47214ba0da65 None None] MFA Rules not processed for user `944700c14fb741e2b3370e777adb76e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:31:56.128823 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-57572f93-6d88-4d96-94f5-47214ba0da65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:31:56.160218 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:31:56.200393 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1360/2694] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:31:56 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:31:56.212239 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff51a73-3a61-4e8b-aa1d-cb3f8c432bce admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:56.213941 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-bff51a73-3a61-4e8b-aa1d-cb3f8c432bce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:31:56.219627 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:31:56.274986 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff51a73-3a61-4e8b-aa1d-cb3f8c432bce admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:56.278629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3473531-262e-4317-b06b-9fbc6860c336 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:56.279244 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3473531-262e-4317-b06b-9fbc6860c336 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:56.279526 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3473531-262e-4317-b06b-9fbc6860c336 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:56.279802 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3473531-262e-4317-b06b-9fbc6860c336 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:56.288929 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3473531-262e-4317-b06b-9fbc6860c336 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:56.290394 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3473531-262e-4317-b06b-9fbc6860c336 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:56.299688 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1335/2695] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:31:56 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:31:56.379674 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57572f93-6d88-4d96-94f5-47214ba0da65 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:31:56.389204 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57572f93-6d88-4d96-94f5-47214ba0da65 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:31:56.392761 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff175291-54fb-4f47-98e2-9e2f8a1834e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:31:56.393947 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff175291-54fb-4f47-98e2-9e2f8a1834e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:31:56.394387 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff175291-54fb-4f47-98e2-9e2f8a1834e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:31:56.394661 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff175291-54fb-4f47-98e2-9e2f8a1834e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:31:56.449716 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff175291-54fb-4f47-98e2-9e2f8a1834e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:31:56.451228 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff175291-54fb-4f47-98e2-9e2f8a1834e6 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:31:56.460772 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1361/2696] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:31:56 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:32:05.604731 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3473531-262e-4317-b06b-9fbc6860c336 service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:05.666256 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3473531-262e-4317-b06b-9fbc6860c336 service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:05.670805 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e5ba5-ebe4-402e-8c8e-462259e42091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:05.671445 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e5ba5-ebe4-402e-8c8e-462259e42091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:05.671727 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e5ba5-ebe4-402e-8c8e-462259e42091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:05.672375 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e5ba5-ebe4-402e-8c8e-462259e42091 admin admin] PATH_INFO: `/v3/users/74a14d5ed67c450599b447c3b08d08ae` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:05.673171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31e5ba5-ebe4-402e-8c8e-462259e42091 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74a14d5ed67c450599b447c3b08d08ae)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:05.675107 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31e5ba5-ebe4-402e-8c8e-462259e42091 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:05.722904 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f31e5ba5-ebe4-402e-8c8e-462259e42091 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74a14d5ed67c450599b447c3b08d08ae'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:05.723674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f31e5ba5-ebe4-402e-8c8e-462259e42091 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74a14d5ed67c450599b447c3b08d08ae'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:05.730900 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f31e5ba5-ebe4-402e-8c8e-462259e42091 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74a14d5ed67c450599b447c3b08d08ae'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:05.743598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f31e5ba5-ebe4-402e-8c8e-462259e42091 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74a14d5ed67c450599b447c3b08d08ae'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:05.750406 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f31e5ba5-ebe4-402e-8c8e-462259e42091 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74a14d5ed67c450599b447c3b08d08ae'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:05.757066 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1336/2697] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:32:05 2022] DELETE /identity/v3/users/74a14d5ed67c450599b447c3b08d08ae => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:32:05.761536 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081b784-c9ff-4f28-9341-06776f6eedc2 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:05.762055 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081b784-c9ff-4f28-9341-06776f6eedc2 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:05.762170 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081b784-c9ff-4f28-9341-06776f6eedc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:05.780456 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-6081b784-c9ff-4f28-9341-06776f6eedc2 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:32:05.781656 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-6081b784-c9ff-4f28-9341-06776f6eedc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:05.839273 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1362/2698] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:32:05.849743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e5ba5-ebe4-402e-8c8e-462259e42091 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:05.851155 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-f31e5ba5-ebe4-402e-8c8e-462259e42091 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:05.878667 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:32:05.909089 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e5ba5-ebe4-402e-8c8e-462259e42091 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:05.913236 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89976c-6863-42d7-a1af-0f061e3e0649 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:05.913844 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89976c-6863-42d7-a1af-0f061e3e0649 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:05.914158 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89976c-6863-42d7-a1af-0f061e3e0649 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:05.914429 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89976c-6863-42d7-a1af-0f061e3e0649 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:05.923245 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c89976c-6863-42d7-a1af-0f061e3e0649 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:05.924662 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c89976c-6863-42d7-a1af-0f061e3e0649 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:05.934580 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1337/2699] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:32:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:32:06.203162 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6081b784-c9ff-4f28-9341-06776f6eedc2 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:06.204621 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-6081b784-c9ff-4f28-9341-06776f6eedc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:06.216798 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6081b784-c9ff-4f28-9341-06776f6eedc2 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:06.221442 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55f37f-16d7-4001-81cd-4bca66645f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:06.222149 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55f37f-16d7-4001-81cd-4bca66645f6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:06.222440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55f37f-16d7-4001-81cd-4bca66645f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:06.222716 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55f37f-16d7-4001-81cd-4bca66645f6b admin admin] PATH_INFO: `/v3/projects/aa498b2b2c0a4b72a9f0e70b4a3e8442` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:06.224399 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e55f37f-16d7-4001-81cd-4bca66645f6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa498b2b2c0a4b72a9f0e70b4a3e8442)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:06.226354 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e55f37f-16d7-4001-81cd-4bca66645f6b admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:06.272269 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9e55f37f-16d7-4001-81cd-4bca66645f6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa498b2b2c0a4b72a9f0e70b4a3e8442', 'request_id': 'req-9e55f37f-16d7-4001-81cd-4bca66645f6b'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:06.278337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9e55f37f-16d7-4001-81cd-4bca66645f6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa498b2b2c0a4b72a9f0e70b4a3e8442', 'request_id': 'req-9e55f37f-16d7-4001-81cd-4bca66645f6b'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:06.282111 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:32:06.284326 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9e55f37f-16d7-4001-81cd-4bca66645f6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa498b2b2c0a4b72a9f0e70b4a3e8442', 'request_id': 'req-9e55f37f-16d7-4001-81cd-4bca66645f6b'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:06.298560 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9e55f37f-16d7-4001-81cd-4bca66645f6b admin admin] The token cache is being invalidate because project aa498b2b2c0a4b72a9f0e70b4a3e8442 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:32:06.298850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9e55f37f-16d7-4001-81cd-4bca66645f6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:06.299473 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-9e55f37f-16d7-4001-81cd-4bca66645f6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:06.301016 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1363/2700] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:32:06 2022] DELETE /identity/v3/projects/aa498b2b2c0a4b72a9f0e70b4a3e8442 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:32:06.305123 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89976c-6863-42d7-a1af-0f061e3e0649 service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:06.314662 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89976c-6863-42d7-a1af-0f061e3e0649 service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:06.375081 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89066a73-92a8-4ea4-a0e3-f4db31978d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:06.375723 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89066a73-92a8-4ea4-a0e3-f4db31978d61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:06.376008 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89066a73-92a8-4ea4-a0e3-f4db31978d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:06.376287 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89066a73-92a8-4ea4-a0e3-f4db31978d61 admin admin] PATH_INFO: `/v3/users/944700c14fb741e2b3370e777adb76e2` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:06.377457 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89066a73-92a8-4ea4-a0e3-f4db31978d61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=944700c14fb741e2b3370e777adb76e2)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:06.379530 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89066a73-92a8-4ea4-a0e3-f4db31978d61 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:06.425559 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-89066a73-92a8-4ea4-a0e3-f4db31978d61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '944700c14fb741e2b3370e777adb76e2'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:06.426626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-89066a73-92a8-4ea4-a0e3-f4db31978d61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '944700c14fb741e2b3370e777adb76e2'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:06.434684 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-89066a73-92a8-4ea4-a0e3-f4db31978d61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '944700c14fb741e2b3370e777adb76e2'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:06.446476 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-89066a73-92a8-4ea4-a0e3-f4db31978d61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '944700c14fb741e2b3370e777adb76e2'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:06.453512 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-89066a73-92a8-4ea4-a0e3-f4db31978d61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '944700c14fb741e2b3370e777adb76e2'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:06.460374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1338/2701] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:32:06 2022] DELETE /identity/v3/users/944700c14fb741e2b3370e777adb76e2 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:32:06.734901 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55f37f-16d7-4001-81cd-4bca66645f6b admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:06.736481 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-9e55f37f-16d7-4001-81cd-4bca66645f6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:06.751605 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55f37f-16d7-4001-81cd-4bca66645f6b admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:06.808612 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70dfee3-0324-4e49-a012-1ee52ccbadb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:06.809255 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70dfee3-0324-4e49-a012-1ee52ccbadb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:06.809446 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70dfee3-0324-4e49-a012-1ee52ccbadb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:06.809724 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70dfee3-0324-4e49-a012-1ee52ccbadb6 admin admin] PATH_INFO: `/v3/projects/f7d9454ca1b94105bed533995bda4a14` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:06.810910 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70dfee3-0324-4e49-a012-1ee52ccbadb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7d9454ca1b94105bed533995bda4a14)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:06.812891 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70dfee3-0324-4e49-a012-1ee52ccbadb6 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:06.858523 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c70dfee3-0324-4e49-a012-1ee52ccbadb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7d9454ca1b94105bed533995bda4a14', 'request_id': 'req-c70dfee3-0324-4e49-a012-1ee52ccbadb6'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:06.864525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c70dfee3-0324-4e49-a012-1ee52ccbadb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7d9454ca1b94105bed533995bda4a14', 'request_id': 'req-c70dfee3-0324-4e49-a012-1ee52ccbadb6'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:06.869860 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c70dfee3-0324-4e49-a012-1ee52ccbadb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7d9454ca1b94105bed533995bda4a14', 'request_id': 'req-c70dfee3-0324-4e49-a012-1ee52ccbadb6'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:06.882253 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c70dfee3-0324-4e49-a012-1ee52ccbadb6 admin admin] The token cache is being invalidate because project f7d9454ca1b94105bed533995bda4a14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:32:06.882545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c70dfee3-0324-4e49-a012-1ee52ccbadb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:06.883172 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c70dfee3-0324-4e49-a012-1ee52ccbadb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:06.884658 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1364/2702] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:32:06 2022] DELETE /identity/v3/projects/f7d9454ca1b94105bed533995bda4a14 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:32:06.889564 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e75aa-349e-49eb-b9ba-fc761494211f None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:06.890059 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e75aa-349e-49eb-b9ba-fc761494211f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:06.890172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e75aa-349e-49eb-b9ba-fc761494211f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:06.911211 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-3d6e75aa-349e-49eb-b9ba-fc761494211f None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:32:06.912416 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3d6e75aa-349e-49eb-b9ba-fc761494211f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:06.954543 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:32:06.973214 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1339/2703] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:32:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:32:06.982066 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafedb6-a373-47d0-9b4e-6c8e7c538c0d None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:06.982353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafedb6-a373-47d0-9b4e-6c8e7c538c0d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:06.982630 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafedb6-a373-47d0-9b4e-6c8e7c538c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:07.002158 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-eeafedb6-a373-47d0-9b4e-6c8e7c538c0d None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:32:07.003364 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-eeafedb6-a373-47d0-9b4e-6c8e7c538c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:07.013035 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1365/2704] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:32:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:32:07.017149 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e75aa-349e-49eb-b9ba-fc761494211f None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:07.026989 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e75aa-349e-49eb-b9ba-fc761494211f None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:07.031669 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03d692-1b31-425a-98ce-d9ca9ce2a06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:07.032283 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03d692-1b31-425a-98ce-d9ca9ce2a06d admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:07.032570 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03d692-1b31-425a-98ce-d9ca9ce2a06d admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:07.032846 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03d692-1b31-425a-98ce-d9ca9ce2a06d admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:07.033543 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03d692-1b31-425a-98ce-d9ca9ce2a06d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:32:07.033848 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03d692-1b31-425a-98ce-d9ca9ce2a06d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:07.035594 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03d692-1b31-425a-98ce-d9ca9ce2a06d admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:07.047016 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1340/2705] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:32:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:32:07.051855 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeafedb6-a373-47d0-9b4e-6c8e7c538c0d None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:07.061353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeafedb6-a373-47d0-9b4e-6c8e7c538c0d None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:07.065960 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027231f-2dd3-4b4e-b4cc-b1686794d3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:07.067201 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027231f-2dd3-4b4e-b4cc-b1686794d3c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:07.067549 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027231f-2dd3-4b4e-b4cc-b1686794d3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:07.067866 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027231f-2dd3-4b4e-b4cc-b1686794d3c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:07.068531 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6027231f-2dd3-4b4e-b4cc-b1686794d3c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:07.070450 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6027231f-2dd3-4b4e-b4cc-b1686794d3c7 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:07.096846 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1366/2706] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:32:07 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:32:07.101101 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03d692-1b31-425a-98ce-d9ca9ce2a06d admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:07.110938 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03d692-1b31-425a-98ce-d9ca9ce2a06d admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:07.167227 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792ce90-c297-40ce-a8b6-59958caeabbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:07.167835 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792ce90-c297-40ce-a8b6-59958caeabbd admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:07.168115 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792ce90-c297-40ce-a8b6-59958caeabbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:07.168386 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792ce90-c297-40ce-a8b6-59958caeabbd admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:07.169039 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e792ce90-c297-40ce-a8b6-59958caeabbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:07.170984 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e792ce90-c297-40ce-a8b6-59958caeabbd admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:07.190952 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1341/2707] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:32:07 2022] POST /identity/v3/users => generated 284 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:32:07.195356 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027231f-2dd3-4b4e-b4cc-b1686794d3c7 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:07.205501 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027231f-2dd3-4b4e-b4cc-b1686794d3c7 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:07.210587 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd42e8c0-8a5b-4349-b9d3-121b63995b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:07.211186 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42e8c0-8a5b-4349-b9d3-121b63995b22 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:07.211480 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42e8c0-8a5b-4349-b9d3-121b63995b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:07.211754 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42e8c0-8a5b-4349-b9d3-121b63995b22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:07.212439 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd42e8c0-8a5b-4349-b9d3-121b63995b22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:07.214017 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd42e8c0-8a5b-4349-b9d3-121b63995b22 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:07.224274 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1367/2708] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:32:07.228024 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792ce90-c297-40ce-a8b6-59958caeabbd admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:07.237745 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792ce90-c297-40ce-a8b6-59958caeabbd admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:07.242895 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7010de52-fead-42ba-99f5-b11a57431ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:07.243497 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010de52-fead-42ba-99f5-b11a57431ab8 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:07.243780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010de52-fead-42ba-99f5-b11a57431ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:07.244055 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010de52-fead-42ba-99f5-b11a57431ab8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:07.244749 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7010de52-fead-42ba-99f5-b11a57431ab8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:07.246391 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7010de52-fead-42ba-99f5-b11a57431ab8 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:07.257181 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1342/2709] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:32:07.261323 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd42e8c0-8a5b-4349-b9d3-121b63995b22 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:07.271652 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd42e8c0-8a5b-4349-b9d3-121b63995b22 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:07.276173 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214693e2-b598-4b14-87c5-67677c21eaf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:07.276825 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214693e2-b598-4b14-87c5-67677c21eaf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:07.277105 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214693e2-b598-4b14-87c5-67677c21eaf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:07.277379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214693e2-b598-4b14-87c5-67677c21eaf8 admin admin] PATH_INFO: `/v3/projects/b467d580c18b4739893d335768c4d5a7/users/e67503e04e3f4df3872aa9f89a23c8ae/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:07.292424 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214693e2-b598-4b14-87c5-67677c21eaf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b467d580c18b4739893d335768c4d5a7, user_id=e67503e04e3f4df3872aa9f89a23c8ae, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:07.295072 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214693e2-b598-4b14-87c5-67677c21eaf8 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:07.307814 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1368/2710] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:32:07 2022] PUT /identity/v3/projects/b467d580c18b4739893d335768c4d5a7/users/e67503e04e3f4df3872aa9f89a23c8ae/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:32:07.318495 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be412f8-093b-42ea-820f-67bccb079f6f None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:07.318717 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be412f8-093b-42ea-820f-67bccb079f6f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:07.318990 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be412f8-093b-42ea-820f-67bccb079f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:07.347462 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-0be412f8-093b-42ea-820f-67bccb079f6f None None] MFA Rules not processed for user `e67503e04e3f4df3872aa9f89a23c8ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:32:07.349285 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-0be412f8-093b-42ea-820f-67bccb079f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:07.410314 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1343/2711] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:32:07 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:32:07.414890 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214693e2-b598-4b14-87c5-67677c21eaf8 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:07.427759 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:32:07.477813 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214693e2-b598-4b14-87c5-67677c21eaf8 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:07.482513 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ce7c2-514b-4886-98c3-dd51f5ad381e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:07.483137 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ce7c2-514b-4886-98c3-dd51f5ad381e admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:07.483419 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ce7c2-514b-4886-98c3-dd51f5ad381e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:07.483695 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ce7c2-514b-4886-98c3-dd51f5ad381e admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:07.484593 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ce7c2-514b-4886-98c3-dd51f5ad381e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:07.487471 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ce7c2-514b-4886-98c3-dd51f5ad381e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:07.512807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1369/2712] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:32:07 2022] POST /identity/v3/projects => generated 367 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:32:07.516829 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be412f8-093b-42ea-820f-67bccb079f6f None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:07.528049 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be412f8-093b-42ea-820f-67bccb079f6f None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:07.568470 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:32:07.586744 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db38388a-bb36-46f8-aa5c-9344497f5e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:07.587366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db38388a-bb36-46f8-aa5c-9344497f5e94 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:07.587646 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db38388a-bb36-46f8-aa5c-9344497f5e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:07.587919 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db38388a-bb36-46f8-aa5c-9344497f5e94 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:07.588598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db38388a-bb36-46f8-aa5c-9344497f5e94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:07.590535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db38388a-bb36-46f8-aa5c-9344497f5e94 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:07.612399 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1344/2713] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:32:07 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:32:07.616531 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ce7c2-514b-4886-98c3-dd51f5ad381e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:07.626848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ce7c2-514b-4886-98c3-dd51f5ad381e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:07.632284 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85972b3-66f3-4aef-9b6f-527253e417ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:07.632895 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85972b3-66f3-4aef-9b6f-527253e417ce admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:07.633178 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85972b3-66f3-4aef-9b6f-527253e417ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:07.633456 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85972b3-66f3-4aef-9b6f-527253e417ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:07.634227 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85972b3-66f3-4aef-9b6f-527253e417ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:07.635815 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85972b3-66f3-4aef-9b6f-527253e417ce admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:07.646865 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1370/2714] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:32:07.650966 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db38388a-bb36-46f8-aa5c-9344497f5e94 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:07.660625 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db38388a-bb36-46f8-aa5c-9344497f5e94 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:07.665223 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab5a45a-f664-49f8-ac4e-e7ba9700cae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:07.665873 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab5a45a-f664-49f8-ac4e-e7ba9700cae6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:07.666184 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab5a45a-f664-49f8-ac4e-e7ba9700cae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:07.666457 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab5a45a-f664-49f8-ac4e-e7ba9700cae6 admin admin] PATH_INFO: `/v3/projects/253ef32d7c3b47a4a056afc2ec114521/users/7acae60be989433bb8aab89b73c92b35/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:07.680732 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab5a45a-f664-49f8-ac4e-e7ba9700cae6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=253ef32d7c3b47a4a056afc2ec114521, user_id=7acae60be989433bb8aab89b73c92b35, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:07.683386 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab5a45a-f664-49f8-ac4e-e7ba9700cae6 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:07.699477 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1345/2715] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:32:07 2022] PUT /identity/v3/projects/253ef32d7c3b47a4a056afc2ec114521/users/7acae60be989433bb8aab89b73c92b35/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:32:07.703473 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85972b3-66f3-4aef-9b6f-527253e417ce admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:07.714224 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85972b3-66f3-4aef-9b6f-527253e417ce admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:07.769269 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fbaea1-04c9-47a3-b4a8-cd598e346473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:07.770119 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fbaea1-04c9-47a3-b4a8-cd598e346473 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:07.770412 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fbaea1-04c9-47a3-b4a8-cd598e346473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:07.770686 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fbaea1-04c9-47a3-b4a8-cd598e346473 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:07.771374 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fbaea1-04c9-47a3-b4a8-cd598e346473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:07.772952 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fbaea1-04c9-47a3-b4a8-cd598e346473 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:07.782365 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1371/2716] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:32:07.787017 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab5a45a-f664-49f8-ac4e-e7ba9700cae6 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:07.796949 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab5a45a-f664-49f8-ac4e-e7ba9700cae6 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:07.802040 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c868e-5107-40bd-87c5-1b55f6e765ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:07.802698 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c868e-5107-40bd-87c5-1b55f6e765ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:07.802976 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c868e-5107-40bd-87c5-1b55f6e765ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:07.803245 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c868e-5107-40bd-87c5-1b55f6e765ea admin admin] PATH_INFO: `/v3/projects/253ef32d7c3b47a4a056afc2ec114521/users/7acae60be989433bb8aab89b73c92b35/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:07.805164 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c868e-5107-40bd-87c5-1b55f6e765ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=253ef32d7c3b47a4a056afc2ec114521, user_id=7acae60be989433bb8aab89b73c92b35, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:07.807805 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c868e-5107-40bd-87c5-1b55f6e765ea admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:07.817844 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1346/2717] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:32:07 2022] PUT /identity/v3/projects/253ef32d7c3b47a4a056afc2ec114521/users/7acae60be989433bb8aab89b73c92b35/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:32:07.828188 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27beb008-6498-4a95-a235-21fe1f7fd77c None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:07.828645 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27beb008-6498-4a95-a235-21fe1f7fd77c None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:07.828758 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27beb008-6498-4a95-a235-21fe1f7fd77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:07.856427 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-27beb008-6498-4a95-a235-21fe1f7fd77c None None] MFA Rules not processed for user `7acae60be989433bb8aab89b73c92b35`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:32:07.857564 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-27beb008-6498-4a95-a235-21fe1f7fd77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:07.924773 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1372/2718] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:32:07 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:32:07.936425 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c868e-5107-40bd-87c5-1b55f6e765ea admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:07.938156 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-dd2c868e-5107-40bd-87c5-1b55f6e765ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:07.994850 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c868e-5107-40bd-87c5-1b55f6e765ea admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:07.998453 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd9053-bd78-4b25-b0b4-157f9bd9411f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:07.999061 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd9053-bd78-4b25-b0b4-157f9bd9411f service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:07.999343 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd9053-bd78-4b25-b0b4-157f9bd9411f service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:07.999618 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd9053-bd78-4b25-b0b4-157f9bd9411f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:08.009059 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd9053-bd78-4b25-b0b4-157f9bd9411f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:08.010514 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd9053-bd78-4b25-b0b4-157f9bd9411f service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:08.019731 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1347/2719] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:32:07 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:32:08.064296 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27beb008-6498-4a95-a235-21fe1f7fd77c None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:08.065867 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-27beb008-6498-4a95-a235-21fe1f7fd77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:08.078412 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27beb008-6498-4a95-a235-21fe1f7fd77c None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:08.081937 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed515403-ac8a-415a-8813-202662d50348 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:08.082754 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed515403-ac8a-415a-8813-202662d50348 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:08.083032 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed515403-ac8a-415a-8813-202662d50348 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:08.083305 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed515403-ac8a-415a-8813-202662d50348 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:08.088357 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:32:08.140390 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed515403-ac8a-415a-8813-202662d50348 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:08.141818 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed515403-ac8a-415a-8813-202662d50348 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:08.150643 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1373/2720] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:32:08 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:32:11.432276 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd9053-bd78-4b25-b0b4-157f9bd9411f service cinder] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:11.492994 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd9053-bd78-4b25-b0b4-157f9bd9411f service cinder] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:11.497470 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e850c0-62d8-4fd2-a6b4-a284f0db68aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:11.498134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e850c0-62d8-4fd2-a6b4-a284f0db68aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:11.498414 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e850c0-62d8-4fd2-a6b4-a284f0db68aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:11.498683 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e850c0-62d8-4fd2-a6b4-a284f0db68aa admin admin] PATH_INFO: `/v3/users/e67503e04e3f4df3872aa9f89a23c8ae` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:11.499819 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e850c0-62d8-4fd2-a6b4-a284f0db68aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e67503e04e3f4df3872aa9f89a23c8ae)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:11.501715 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e850c0-62d8-4fd2-a6b4-a284f0db68aa admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:11.541636 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:32:11.548334 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-02e850c0-62d8-4fd2-a6b4-a284f0db68aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e67503e04e3f4df3872aa9f89a23c8ae'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:11.549397 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-02e850c0-62d8-4fd2-a6b4-a284f0db68aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e67503e04e3f4df3872aa9f89a23c8ae'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:11.557014 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-02e850c0-62d8-4fd2-a6b4-a284f0db68aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e67503e04e3f4df3872aa9f89a23c8ae'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:11.569604 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-02e850c0-62d8-4fd2-a6b4-a284f0db68aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e67503e04e3f4df3872aa9f89a23c8ae'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:11.576613 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-02e850c0-62d8-4fd2-a6b4-a284f0db68aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e67503e04e3f4df3872aa9f89a23c8ae'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:11.582908 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1348/2721] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:32:11 2022] DELETE /identity/v3/users/e67503e04e3f4df3872aa9f89a23c8ae => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:32:11.587498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0deaaf0-cea0-494f-b018-de2d6be784b5 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:11.587982 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0deaaf0-cea0-494f-b018-de2d6be784b5 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:11.588096 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0deaaf0-cea0-494f-b018-de2d6be784b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:11.607332 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-d0deaaf0-cea0-494f-b018-de2d6be784b5 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:32:11.608515 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d0deaaf0-cea0-494f-b018-de2d6be784b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:11.667753 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1374/2722] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:32:11.677722 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e850c0-62d8-4fd2-a6b4-a284f0db68aa admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:11.679233 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-02e850c0-62d8-4fd2-a6b4-a284f0db68aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:11.736205 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e850c0-62d8-4fd2-a6b4-a284f0db68aa admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:11.739950 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54352325-3735-4b66-9f2b-60c914b302f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:11.740569 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54352325-3735-4b66-9f2b-60c914b302f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:11.740852 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54352325-3735-4b66-9f2b-60c914b302f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:11.741129 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54352325-3735-4b66-9f2b-60c914b302f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:11.750105 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54352325-3735-4b66-9f2b-60c914b302f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:11.751532 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54352325-3735-4b66-9f2b-60c914b302f0 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:11.761034 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1349/2723] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:32:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:32:12.029580 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0deaaf0-cea0-494f-b018-de2d6be784b5 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:12.031133 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d0deaaf0-cea0-494f-b018-de2d6be784b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:12.044163 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0deaaf0-cea0-494f-b018-de2d6be784b5 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:12.049356 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d0995-fd35-4e82-b4d8-ff9ed2acc7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:12.050024 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d0995-fd35-4e82-b4d8-ff9ed2acc7a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:12.050306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d0995-fd35-4e82-b4d8-ff9ed2acc7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:12.050602 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d0995-fd35-4e82-b4d8-ff9ed2acc7a7 admin admin] PATH_INFO: `/v3/projects/b467d580c18b4739893d335768c4d5a7` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:12.051933 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696d0995-fd35-4e82-b4d8-ff9ed2acc7a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b467d580c18b4739893d335768c4d5a7)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:12.053916 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696d0995-fd35-4e82-b4d8-ff9ed2acc7a7 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:12.099917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-696d0995-fd35-4e82-b4d8-ff9ed2acc7a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b467d580c18b4739893d335768c4d5a7', 'request_id': 'req-696d0995-fd35-4e82-b4d8-ff9ed2acc7a7'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:12.105827 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-696d0995-fd35-4e82-b4d8-ff9ed2acc7a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b467d580c18b4739893d335768c4d5a7', 'request_id': 'req-696d0995-fd35-4e82-b4d8-ff9ed2acc7a7'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:12.111072 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-696d0995-fd35-4e82-b4d8-ff9ed2acc7a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b467d580c18b4739893d335768c4d5a7', 'request_id': 'req-696d0995-fd35-4e82-b4d8-ff9ed2acc7a7'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:12.122696 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-696d0995-fd35-4e82-b4d8-ff9ed2acc7a7 admin admin] The token cache is being invalidate because project b467d580c18b4739893d335768c4d5a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:32:12.122982 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-696d0995-fd35-4e82-b4d8-ff9ed2acc7a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:12.123593 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-696d0995-fd35-4e82-b4d8-ff9ed2acc7a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:12.125057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1375/2724] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:32:12 2022] DELETE /identity/v3/projects/b467d580c18b4739893d335768c4d5a7 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:32:12.129719 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54352325-3735-4b66-9f2b-60c914b302f0 service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:12.139561 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54352325-3735-4b66-9f2b-60c914b302f0 service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:12.195983 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b16c2c-d105-4b06-9ce9-befecc3747b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:12.196632 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b16c2c-d105-4b06-9ce9-befecc3747b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:12.196913 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b16c2c-d105-4b06-9ce9-befecc3747b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:12.197196 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b16c2c-d105-4b06-9ce9-befecc3747b0 admin admin] PATH_INFO: `/v3/users/7acae60be989433bb8aab89b73c92b35` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:12.198642 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b16c2c-d105-4b06-9ce9-befecc3747b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7acae60be989433bb8aab89b73c92b35)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:12.200565 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b16c2c-d105-4b06-9ce9-befecc3747b0 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:12.245755 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-e2b16c2c-d105-4b06-9ce9-befecc3747b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7acae60be989433bb8aab89b73c92b35'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:12.246446 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-e2b16c2c-d105-4b06-9ce9-befecc3747b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7acae60be989433bb8aab89b73c92b35'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:12.253531 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-e2b16c2c-d105-4b06-9ce9-befecc3747b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7acae60be989433bb8aab89b73c92b35'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:12.259634 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:32:12.268631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-e2b16c2c-d105-4b06-9ce9-befecc3747b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7acae60be989433bb8aab89b73c92b35'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:12.276335 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-e2b16c2c-d105-4b06-9ce9-befecc3747b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7acae60be989433bb8aab89b73c92b35'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:12.282148 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1350/2725] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:32:12 2022] DELETE /identity/v3/users/7acae60be989433bb8aab89b73c92b35 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:32:12.563198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d0995-fd35-4e82-b4d8-ff9ed2acc7a7 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:12.564711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-696d0995-fd35-4e82-b4d8-ff9ed2acc7a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:12.578602 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d0995-fd35-4e82-b4d8-ff9ed2acc7a7 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:12.608696 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:32:12.638022 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549401ea-13e9-4527-ac48-da2b6405fc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:12.638669 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549401ea-13e9-4527-ac48-da2b6405fc35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:12.638953 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549401ea-13e9-4527-ac48-da2b6405fc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:12.639277 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549401ea-13e9-4527-ac48-da2b6405fc35 admin admin] PATH_INFO: `/v3/projects/253ef32d7c3b47a4a056afc2ec114521` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:12.640382 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549401ea-13e9-4527-ac48-da2b6405fc35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=253ef32d7c3b47a4a056afc2ec114521)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:12.642309 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549401ea-13e9-4527-ac48-da2b6405fc35 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:12.692533 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-549401ea-13e9-4527-ac48-da2b6405fc35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '253ef32d7c3b47a4a056afc2ec114521', 'request_id': 'req-549401ea-13e9-4527-ac48-da2b6405fc35'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:12.698884 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-549401ea-13e9-4527-ac48-da2b6405fc35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '253ef32d7c3b47a4a056afc2ec114521', 'request_id': 'req-549401ea-13e9-4527-ac48-da2b6405fc35'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:12.704286 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-549401ea-13e9-4527-ac48-da2b6405fc35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '253ef32d7c3b47a4a056afc2ec114521', 'request_id': 'req-549401ea-13e9-4527-ac48-da2b6405fc35'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:12.717771 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-549401ea-13e9-4527-ac48-da2b6405fc35 admin admin] The token cache is being invalidate because project 253ef32d7c3b47a4a056afc2ec114521 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:32:12.718089 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-549401ea-13e9-4527-ac48-da2b6405fc35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:12.718736 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-549401ea-13e9-4527-ac48-da2b6405fc35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:12.720219 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1376/2726] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:32:12 2022] DELETE /identity/v3/projects/253ef32d7c3b47a4a056afc2ec114521 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:32:12.725235 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919b60a-20f7-4f00-a19b-8c21b5c7324f None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:12.725689 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919b60a-20f7-4f00-a19b-8c21b5c7324f None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:12.725808 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919b60a-20f7-4f00-a19b-8c21b5c7324f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:12.745159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-a919b60a-20f7-4f00-a19b-8c21b5c7324f None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:32:12.746394 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-a919b60a-20f7-4f00-a19b-8c21b5c7324f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:12.804252 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1351/2727] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:32:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:32:12.812220 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4687a31e-b9f8-45a8-bbeb-caa90f7ebfda None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:12.812509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4687a31e-b9f8-45a8-bbeb-caa90f7ebfda None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:12.812786 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4687a31e-b9f8-45a8-bbeb-caa90f7ebfda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:12.833399 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-4687a31e-b9f8-45a8-bbeb-caa90f7ebfda None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:32:12.834645 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-4687a31e-b9f8-45a8-bbeb-caa90f7ebfda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:12.842498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1377/2728] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:32:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:32:12.846708 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a919b60a-20f7-4f00-a19b-8c21b5c7324f None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:12.856600 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a919b60a-20f7-4f00-a19b-8c21b5c7324f None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:12.861160 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6415bb39-84bd-49e1-b959-c409d1903deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:12.861744 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415bb39-84bd-49e1-b959-c409d1903deb admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:12.862047 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415bb39-84bd-49e1-b959-c409d1903deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:12.862323 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415bb39-84bd-49e1-b959-c409d1903deb admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:12.863014 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6415bb39-84bd-49e1-b959-c409d1903deb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:32:12.863360 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6415bb39-84bd-49e1-b959-c409d1903deb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:12.865118 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6415bb39-84bd-49e1-b959-c409d1903deb admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:12.876659 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1352/2729] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:32:12.881403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4687a31e-b9f8-45a8-bbeb-caa90f7ebfda None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:12.890520 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4687a31e-b9f8-45a8-bbeb-caa90f7ebfda None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:12.895705 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572093c5-1919-4363-be46-d5d88bfe7459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:12.896313 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572093c5-1919-4363-be46-d5d88bfe7459 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:12.896588 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572093c5-1919-4363-be46-d5d88bfe7459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:12.896853 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572093c5-1919-4363-be46-d5d88bfe7459 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:12.897509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572093c5-1919-4363-be46-d5d88bfe7459 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:12.899397 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572093c5-1919-4363-be46-d5d88bfe7459 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:12.925417 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1378/2730] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:32:12 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:32:12.930192 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6415bb39-84bd-49e1-b959-c409d1903deb admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:12.940392 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6415bb39-84bd-49e1-b959-c409d1903deb admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:12.995474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db581b1-0f10-4237-ac69-c57c1fd14352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:12.996083 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db581b1-0f10-4237-ac69-c57c1fd14352 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:12.996360 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db581b1-0f10-4237-ac69-c57c1fd14352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:12.996631 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db581b1-0f10-4237-ac69-c57c1fd14352 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:12.997297 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db581b1-0f10-4237-ac69-c57c1fd14352 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:12.999226 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db581b1-0f10-4237-ac69-c57c1fd14352 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:13.019191 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1353/2731] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:32:12 2022] POST /identity/v3/users => generated 286 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:32:13.023948 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572093c5-1919-4363-be46-d5d88bfe7459 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:13.033874 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572093c5-1919-4363-be46-d5d88bfe7459 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:13.039441 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06edd20-b4a5-40ef-b4e4-353fa5112e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:13.040045 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06edd20-b4a5-40ef-b4e4-353fa5112e6b admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:13.040323 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06edd20-b4a5-40ef-b4e4-353fa5112e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:13.040591 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06edd20-b4a5-40ef-b4e4-353fa5112e6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:13.041279 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06edd20-b4a5-40ef-b4e4-353fa5112e6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:13.042885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06edd20-b4a5-40ef-b4e4-353fa5112e6b admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:13.052814 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1379/2732] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:32:13.057066 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db581b1-0f10-4237-ac69-c57c1fd14352 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:13.067048 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db581b1-0f10-4237-ac69-c57c1fd14352 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:13.071557 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4370eea-90fe-444a-90ab-e6cbd6a5d649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:13.072157 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4370eea-90fe-444a-90ab-e6cbd6a5d649 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:13.072435 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4370eea-90fe-444a-90ab-e6cbd6a5d649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:13.072714 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4370eea-90fe-444a-90ab-e6cbd6a5d649 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:13.073664 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4370eea-90fe-444a-90ab-e6cbd6a5d649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:13.075285 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4370eea-90fe-444a-90ab-e6cbd6a5d649 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:13.087081 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1354/2733] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:32:13.091098 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06edd20-b4a5-40ef-b4e4-353fa5112e6b admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:13.102369 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06edd20-b4a5-40ef-b4e4-353fa5112e6b admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:13.106976 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88c953-2672-44aa-9c8e-e139a599b453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:13.107632 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88c953-2672-44aa-9c8e-e139a599b453 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:13.107916 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88c953-2672-44aa-9c8e-e139a599b453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:13.108192 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88c953-2672-44aa-9c8e-e139a599b453 admin admin] PATH_INFO: `/v3/projects/f8386eccb5254b5591e63df14e2fe6e9/users/4e9f70da3fc24aa08a18b96e2d7ea2c0/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:13.123218 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d88c953-2672-44aa-9c8e-e139a599b453 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8386eccb5254b5591e63df14e2fe6e9, user_id=4e9f70da3fc24aa08a18b96e2d7ea2c0, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:13.125844 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d88c953-2672-44aa-9c8e-e139a599b453 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:13.139831 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1380/2734] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:32:13 2022] PUT /identity/v3/projects/f8386eccb5254b5591e63df14e2fe6e9/users/4e9f70da3fc24aa08a18b96e2d7ea2c0/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:32:13.150771 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d640f-d805-45e6-b1e8-3de7723b6a42 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:13.150996 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d640f-d805-45e6-b1e8-3de7723b6a42 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:13.151272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d640f-d805-45e6-b1e8-3de7723b6a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:13.180016 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-9e8d640f-d805-45e6-b1e8-3de7723b6a42 None None] MFA Rules not processed for user `4e9f70da3fc24aa08a18b96e2d7ea2c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:32:13.181227 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9e8d640f-d805-45e6-b1e8-3de7723b6a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:13.214233 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:32:13.248076 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1355/2735] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:32:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:32:13.259429 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88c953-2672-44aa-9c8e-e139a599b453 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:13.261145 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-9d88c953-2672-44aa-9c8e-e139a599b453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:13.318644 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88c953-2672-44aa-9c8e-e139a599b453 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:13.322276 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1446c76-b08d-43a4-8d0a-2befe15bd68c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:13.322894 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1446c76-b08d-43a4-8d0a-2befe15bd68c service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:13.323440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1446c76-b08d-43a4-8d0a-2befe15bd68c service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:13.323720 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1446c76-b08d-43a4-8d0a-2befe15bd68c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:13.332154 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1446c76-b08d-43a4-8d0a-2befe15bd68c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:13.333577 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1446c76-b08d-43a4-8d0a-2befe15bd68c service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:13.338098 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:32:13.343289 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1381/2736] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:32:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:32:13.365653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d640f-d805-45e6-b1e8-3de7723b6a42 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:13.427225 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d640f-d805-45e6-b1e8-3de7723b6a42 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:13.431769 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d1ed36-633b-4d9c-862b-f4d806ac8c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:13.432406 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1ed36-633b-4d9c-862b-f4d806ac8c60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:13.432689 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1ed36-633b-4d9c-862b-f4d806ac8c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:13.432967 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1ed36-633b-4d9c-862b-f4d806ac8c60 admin admin] PATH_INFO: `/v3/users/4e9f70da3fc24aa08a18b96e2d7ea2c0` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:13.434180 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d1ed36-633b-4d9c-862b-f4d806ac8c60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e9f70da3fc24aa08a18b96e2d7ea2c0)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:13.436082 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d1ed36-633b-4d9c-862b-f4d806ac8c60 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:13.483117 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-21d1ed36-633b-4d9c-862b-f4d806ac8c60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e9f70da3fc24aa08a18b96e2d7ea2c0'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:13.483793 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-21d1ed36-633b-4d9c-862b-f4d806ac8c60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e9f70da3fc24aa08a18b96e2d7ea2c0'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:13.491723 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-21d1ed36-633b-4d9c-862b-f4d806ac8c60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e9f70da3fc24aa08a18b96e2d7ea2c0'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:13.503617 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-21d1ed36-633b-4d9c-862b-f4d806ac8c60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e9f70da3fc24aa08a18b96e2d7ea2c0'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:13.511552 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-21d1ed36-633b-4d9c-862b-f4d806ac8c60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e9f70da3fc24aa08a18b96e2d7ea2c0'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:13.517491 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1356/2737] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:32:13 2022] DELETE /identity/v3/users/4e9f70da3fc24aa08a18b96e2d7ea2c0 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:32:13.522189 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ad375-715c-44b7-9edf-6c1bfb253cab None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:13.522484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ad375-715c-44b7-9edf-6c1bfb253cab None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:13.522765 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ad375-715c-44b7-9edf-6c1bfb253cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:13.542621 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-f43ad375-715c-44b7-9edf-6c1bfb253cab None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:32:13.543826 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f43ad375-715c-44b7-9edf-6c1bfb253cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:13.602285 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1382/2738] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:32:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:32:13.611909 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d1ed36-633b-4d9c-862b-f4d806ac8c60 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:13.613958 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-21d1ed36-633b-4d9c-862b-f4d806ac8c60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:13.670480 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d1ed36-633b-4d9c-862b-f4d806ac8c60 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:13.674049 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7659cb-273f-443c-bf27-0e899b056496 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:13.674658 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7659cb-273f-443c-bf27-0e899b056496 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:13.674935 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7659cb-273f-443c-bf27-0e899b056496 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:13.675208 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7659cb-273f-443c-bf27-0e899b056496 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:13.682527 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:32:13.684780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7659cb-273f-443c-bf27-0e899b056496 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:13.686594 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7659cb-273f-443c-bf27-0e899b056496 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:13.698055 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1357/2739] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:32:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:32:13.967047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ad375-715c-44b7-9edf-6c1bfb253cab None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:13.968599 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f43ad375-715c-44b7-9edf-6c1bfb253cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:13.981628 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ad375-715c-44b7-9edf-6c1bfb253cab None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:13.986488 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5bd05-9e4f-4f82-b27e-2d7243567262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:13.987211 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5bd05-9e4f-4f82-b27e-2d7243567262 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:13.987564 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5bd05-9e4f-4f82-b27e-2d7243567262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:13.988401 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5bd05-9e4f-4f82-b27e-2d7243567262 admin admin] PATH_INFO: `/v3/projects/f8386eccb5254b5591e63df14e2fe6e9` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:13.989821 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f5bd05-9e4f-4f82-b27e-2d7243567262 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8386eccb5254b5591e63df14e2fe6e9)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:13.991762 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f5bd05-9e4f-4f82-b27e-2d7243567262 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:14.038539 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-66f5bd05-9e4f-4f82-b27e-2d7243567262 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8386eccb5254b5591e63df14e2fe6e9', 'request_id': 'req-66f5bd05-9e4f-4f82-b27e-2d7243567262'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:14.044633 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-66f5bd05-9e4f-4f82-b27e-2d7243567262 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8386eccb5254b5591e63df14e2fe6e9', 'request_id': 'req-66f5bd05-9e4f-4f82-b27e-2d7243567262'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:14.050208 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-66f5bd05-9e4f-4f82-b27e-2d7243567262 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8386eccb5254b5591e63df14e2fe6e9', 'request_id': 'req-66f5bd05-9e4f-4f82-b27e-2d7243567262'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:14.062040 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-66f5bd05-9e4f-4f82-b27e-2d7243567262 admin admin] The token cache is being invalidate because project f8386eccb5254b5591e63df14e2fe6e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:32:14.062357 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-66f5bd05-9e4f-4f82-b27e-2d7243567262 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:14.062979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-66f5bd05-9e4f-4f82-b27e-2d7243567262 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:32:14.064464 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1383/2740] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:32:13 2022] DELETE /identity/v3/projects/f8386eccb5254b5591e63df14e2fe6e9 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:32:14.069390 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98feac83-6038-4bb4-9b00-c146a6c85d1c None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:14.069881 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98feac83-6038-4bb4-9b00-c146a6c85d1c None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:14.070014 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98feac83-6038-4bb4-9b00-c146a6c85d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:14.089748 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-98feac83-6038-4bb4-9b00-c146a6c85d1c None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:32:14.090988 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-98feac83-6038-4bb4-9b00-c146a6c85d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:14.154614 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1358/2741] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:32:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:32:14.163494 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6da9ab-163b-426d-bdf0-2fc0788ec755 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:14.163782 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6da9ab-163b-426d-bdf0-2fc0788ec755 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:14.164056 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6da9ab-163b-426d-bdf0-2fc0788ec755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:14.183165 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-2a6da9ab-163b-426d-bdf0-2fc0788ec755 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:32:14.184353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2a6da9ab-163b-426d-bdf0-2fc0788ec755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:14.192680 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1384/2742] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:32:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:32:14.196764 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98feac83-6038-4bb4-9b00-c146a6c85d1c None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:14.206590 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98feac83-6038-4bb4-9b00-c146a6c85d1c None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:14.211748 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f3e84-1214-4881-bccc-59f191dee9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:14.212292 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f3e84-1214-4881-bccc-59f191dee9b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:14.212569 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f3e84-1214-4881-bccc-59f191dee9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:14.212842 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f3e84-1214-4881-bccc-59f191dee9b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:14.213531 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909f3e84-1214-4881-bccc-59f191dee9b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:32:14.213833 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909f3e84-1214-4881-bccc-59f191dee9b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:14.215454 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909f3e84-1214-4881-bccc-59f191dee9b3 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:14.227460 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1359/2743] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:32:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:32:14.231607 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6da9ab-163b-426d-bdf0-2fc0788ec755 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:14.240891 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6da9ab-163b-426d-bdf0-2fc0788ec755 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:14.245526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7b261-d96d-4ed3-bd0a-7f0787c82487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:14.246164 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7b261-d96d-4ed3-bd0a-7f0787c82487 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:14.246443 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7b261-d96d-4ed3-bd0a-7f0787c82487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:14.246714 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7b261-d96d-4ed3-bd0a-7f0787c82487 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:14.247373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b7b261-d96d-4ed3-bd0a-7f0787c82487 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:14.249239 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b7b261-d96d-4ed3-bd0a-7f0787c82487 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:14.277014 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1385/2744] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:32:14 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:32:14.281301 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f3e84-1214-4881-bccc-59f191dee9b3 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:14.291720 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f3e84-1214-4881-bccc-59f191dee9b3 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:14.350180 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52adcfe6-81ca-453c-865d-8eace3ccd60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:14.350806 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52adcfe6-81ca-453c-865d-8eace3ccd60e admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:14.351090 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52adcfe6-81ca-453c-865d-8eace3ccd60e admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:14.351371 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52adcfe6-81ca-453c-865d-8eace3ccd60e admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:14.352062 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52adcfe6-81ca-453c-865d-8eace3ccd60e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:14.353961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52adcfe6-81ca-453c-865d-8eace3ccd60e admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:14.373924 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1360/2745] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:32:14 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:32:14.378065 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7b261-d96d-4ed3-bd0a-7f0787c82487 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:14.389227 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7b261-d96d-4ed3-bd0a-7f0787c82487 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:14.393880 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891952eb-eca0-436e-9c55-4a5b9f8de758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:14.395691 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891952eb-eca0-436e-9c55-4a5b9f8de758 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:14.395956 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891952eb-eca0-436e-9c55-4a5b9f8de758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:14.396235 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891952eb-eca0-436e-9c55-4a5b9f8de758 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:14.396935 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891952eb-eca0-436e-9c55-4a5b9f8de758 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:14.398561 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891952eb-eca0-436e-9c55-4a5b9f8de758 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:14.408477 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1386/2746] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:32:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 04:32:14.412686 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52adcfe6-81ca-453c-865d-8eace3ccd60e admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:14.422365 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52adcfe6-81ca-453c-865d-8eace3ccd60e admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:14.427191 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771d452-8c28-4857-8699-7bb705f9f498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:14.427792 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771d452-8c28-4857-8699-7bb705f9f498 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:14.428071 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771d452-8c28-4857-8699-7bb705f9f498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:14.428344 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771d452-8c28-4857-8699-7bb705f9f498 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:14.429035 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5771d452-8c28-4857-8699-7bb705f9f498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:14.430723 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5771d452-8c28-4857-8699-7bb705f9f498 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:14.440969 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1361/2747] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:32:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:32:14.445534 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891952eb-eca0-436e-9c55-4a5b9f8de758 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:14.455676 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891952eb-eca0-436e-9c55-4a5b9f8de758 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:14.461301 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b02bde-3c33-4538-af5b-348917655a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:14.462047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b02bde-3c33-4538-af5b-348917655a87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:14.462274 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b02bde-3c33-4538-af5b-348917655a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:14.462545 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b02bde-3c33-4538-af5b-348917655a87 admin admin] PATH_INFO: `/v3/projects/37f8c051d2c14448ab9a73672f3f00a9/users/69e9a87bc39f4456ab90586e02d0785d/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:14.477061 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b02bde-3c33-4538-af5b-348917655a87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37f8c051d2c14448ab9a73672f3f00a9, user_id=69e9a87bc39f4456ab90586e02d0785d, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:14.479713 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b02bde-3c33-4538-af5b-348917655a87 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:14.492583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1387/2748] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:32:14 2022] PUT /identity/v3/projects/37f8c051d2c14448ab9a73672f3f00a9/users/69e9a87bc39f4456ab90586e02d0785d/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:32:14.497591 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789553bc-60ce-4285-bf3a-2a2aa72f7456 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:14.497876 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789553bc-60ce-4285-bf3a-2a2aa72f7456 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:14.498198 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789553bc-60ce-4285-bf3a-2a2aa72f7456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:14.517192 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-789553bc-60ce-4285-bf3a-2a2aa72f7456 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:32:14.518407 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-789553bc-60ce-4285-bf3a-2a2aa72f7456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:14.576094 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1362/2749] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:32:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:32:14.586341 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b02bde-3c33-4538-af5b-348917655a87 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:14.588067 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-20b02bde-3c33-4538-af5b-348917655a87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:14.609659 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:32:14.644047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b02bde-3c33-4538-af5b-348917655a87 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:14.648337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5368f253-0f27-4c8a-b3f6-7b3fd5dff329 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:14.648944 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368f253-0f27-4c8a-b3f6-7b3fd5dff329 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:14.649220 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368f253-0f27-4c8a-b3f6-7b3fd5dff329 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:14.649491 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368f253-0f27-4c8a-b3f6-7b3fd5dff329 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:14.658121 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5368f253-0f27-4c8a-b3f6-7b3fd5dff329 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:14.659528 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5368f253-0f27-4c8a-b3f6-7b3fd5dff329 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:14.669472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1388/2750] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:32:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:32:15.278417 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679995b-883f-49a6-b51c-2c24d5853763 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:15.279305 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679995b-883f-49a6-b51c-2c24d5853763 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:15.279615 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679995b-883f-49a6-b51c-2c24d5853763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:15.287491 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:32:15.312101 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-2679995b-883f-49a6-b51c-2c24d5853763 None None] MFA Rules not processed for user `69e9a87bc39f4456ab90586e02d0785d`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:32:15.313422 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2679995b-883f-49a6-b51c-2c24d5853763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:15.380461 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1363/2751] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:32:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:32:15.391200 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5368f253-0f27-4c8a-b3f6-7b3fd5dff329 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:15.392582 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5368f253-0f27-4c8a-b3f6-7b3fd5dff329 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:15.454300 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5368f253-0f27-4c8a-b3f6-7b3fd5dff329 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:15.459716 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d944f0f5-3491-4f61-9a7d-07894de546aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:15.460350 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944f0f5-3491-4f61-9a7d-07894de546aa service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:15.460635 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944f0f5-3491-4f61-9a7d-07894de546aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:15.461361 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944f0f5-3491-4f61-9a7d-07894de546aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:15.470566 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d944f0f5-3491-4f61-9a7d-07894de546aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:15.471992 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d944f0f5-3491-4f61-9a7d-07894de546aa service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:15.481116 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1389/2752] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:32:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:32:16.830322 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565f935-403d-4e12-84be-6e5029a098c0 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:16.830830 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565f935-403d-4e12-84be-6e5029a098c0 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:16.830931 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565f935-403d-4e12-84be-6e5029a098c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:16.850493 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-2565f935-403d-4e12-84be-6e5029a098c0 None None] MFA Rules not processed for user `69e9a87bc39f4456ab90586e02d0785d`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:32:16.851671 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2565f935-403d-4e12-84be-6e5029a098c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:16.858613 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1364/2753] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:32:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:32:17.412559 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d944f0f5-3491-4f61-9a7d-07894de546aa service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:17.414187 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d944f0f5-3491-4f61-9a7d-07894de546aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:17.517378 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d944f0f5-3491-4f61-9a7d-07894de546aa service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:17.523242 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce79d8-8a0f-405a-b51f-9bd7a7fee67d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c08fba2d7de42218400c5e652cc3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:17.523862 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce79d8-8a0f-405a-b51f-9bd7a7fee67d service placement] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:17.524143 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce79d8-8a0f-405a-b51f-9bd7a7fee67d service placement] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:17.524420 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce79d8-8a0f-405a-b51f-9bd7a7fee67d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:17.526290 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-59ce79d8-8a0f-405a-b51f-9bd7a7fee67d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:17.538490 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce79d8-8a0f-405a-b51f-9bd7a7fee67d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:17.540142 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce79d8-8a0f-405a-b51f-9bd7a7fee67d service placement] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:17.549807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1390/2754] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:32:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:32:19.112202 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2565f935-403d-4e12-84be-6e5029a098c0 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:19.114422 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2565f935-403d-4e12-84be-6e5029a098c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:19.126395 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2565f935-403d-4e12-84be-6e5029a098c0 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:19.130037 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb685e-bb72-4275-abf0-38cbfd9d3e0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:19.130675 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb685e-bb72-4275-abf0-38cbfd9d3e0b service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:19.130959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb685e-bb72-4275-abf0-38cbfd9d3e0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:19.131235 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb685e-bb72-4275-abf0-38cbfd9d3e0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:19.132937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e0fb685e-bb72-4275-abf0-38cbfd9d3e0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:19.142344 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb685e-bb72-4275-abf0-38cbfd9d3e0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:19.143810 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb685e-bb72-4275-abf0-38cbfd9d3e0b service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:19.152433 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1365/2755] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:32:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:32:27.545616 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=75 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:32:29.149110 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=40 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:32:31.226392 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c69a7-613d-4392-9ea3-0a504fef8a15 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:31.226896 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c69a7-613d-4392-9ea3-0a504fef8a15 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:31.227017 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c69a7-613d-4392-9ea3-0a504fef8a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:31.246897 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-360c69a7-613d-4392-9ea3-0a504fef8a15 None None] MFA Rules not processed for user `69e9a87bc39f4456ab90586e02d0785d`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:32:31.248129 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-360c69a7-613d-4392-9ea3-0a504fef8a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:31.255673 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1391/2756] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:32:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:32:31.498424 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb685e-bb72-4275-abf0-38cbfd9d3e0b service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:31.508423 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb685e-bb72-4275-abf0-38cbfd9d3e0b service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:31.513173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40858d11-a38e-4299-8baf-1175edb910ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:31.513877 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40858d11-a38e-4299-8baf-1175edb910ee service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:31.514191 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40858d11-a38e-4299-8baf-1175edb910ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:31.514476 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40858d11-a38e-4299-8baf-1175edb910ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:31.516244 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-40858d11-a38e-4299-8baf-1175edb910ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:31.527022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40858d11-a38e-4299-8baf-1175edb910ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:31.528608 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40858d11-a38e-4299-8baf-1175edb910ee service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:31.537013 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1366/2757] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:32:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:32:31.684388 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c2bde-ce8a-490b-ab4a-6a72cc04cf1b None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:31.684679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c2bde-ce8a-490b-ab4a-6a72cc04cf1b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:31.684988 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c2bde-ce8a-490b-ab4a-6a72cc04cf1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:31.705331 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-885c2bde-ce8a-490b-ab4a-6a72cc04cf1b None None] MFA Rules not processed for user `89aa18ddc71c4951bf15fcbd404912e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:32:31.706560 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-885c2bde-ce8a-490b-ab4a-6a72cc04cf1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:31.765000 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1392/2758] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:32:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:32:35.058824 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40858d11-a38e-4299-8baf-1175edb910ee service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:35.060488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-40858d11-a38e-4299-8baf-1175edb910ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:35.142591 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40858d11-a38e-4299-8baf-1175edb910ee service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:35.148379 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04570883-e990-46c4-ae6b-b5843c66b2f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:35.149017 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04570883-e990-46c4-ae6b-b5843c66b2f7 service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:35.149298 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04570883-e990-46c4-ae6b-b5843c66b2f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:35.149572 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04570883-e990-46c4-ae6b-b5843c66b2f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:35.151439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-04570883-e990-46c4-ae6b-b5843c66b2f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:35.166385 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04570883-e990-46c4-ae6b-b5843c66b2f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:35.168053 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04570883-e990-46c4-ae6b-b5843c66b2f7 service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:35.182172 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1367/2759] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:32:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:32:38.892121 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c2bde-ce8a-490b-ab4a-6a72cc04cf1b None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:38.893640 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-885c2bde-ce8a-490b-ab4a-6a72cc04cf1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:38.913538 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c2bde-ce8a-490b-ab4a-6a72cc04cf1b None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:38.919645 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcb3530-cae1-4912-a7e9-83e6671028e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:38.920290 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb3530-cae1-4912-a7e9-83e6671028e5 service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:38.920573 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb3530-cae1-4912-a7e9-83e6671028e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:38.920920 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb3530-cae1-4912-a7e9-83e6671028e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:38.923374 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5bcb3530-cae1-4912-a7e9-83e6671028e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:38.937123 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcb3530-cae1-4912-a7e9-83e6671028e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:38.938743 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcb3530-cae1-4912-a7e9-83e6671028e5 service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:38.951587 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1393/2760] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:32:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:32:45.174785 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=40 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:32:45.539294 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04570883-e990-46c4-ae6b-b5843c66b2f7 service nova] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:45.551422 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04570883-e990-46c4-ae6b-b5843c66b2f7 service nova] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:45.556939 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6ce30-ad5b-4ca7-98df-de69e5a1560f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc9ea8f89c14a2a8e3272f750647dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:45.557555 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6ce30-ad5b-4ca7-98df-de69e5a1560f service nova] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:45.557834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6ce30-ad5b-4ca7-98df-de69e5a1560f service nova] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:45.558139 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6ce30-ad5b-4ca7-98df-de69e5a1560f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:45.560186 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-79f6ce30-ad5b-4ca7-98df-de69e5a1560f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:45.573163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f6ce30-ad5b-4ca7-98df-de69e5a1560f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:45.575068 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f6ce30-ad5b-4ca7-98df-de69e5a1560f service nova] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:45.586854 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1368/2761] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:32:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:32:48.946356 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=34 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:32:48.950287 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d99bb-4ded-4c0d-a0e8-4b09237c0501 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:48.950794 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d99bb-4ded-4c0d-a0e8-4b09237c0501 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:48.951077 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d99bb-4ded-4c0d-a0e8-4b09237c0501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:49.009160 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-957d99bb-4ded-4c0d-a0e8-4b09237c0501 None None] MFA Rules not processed for user `8ec07ee1efba4e8089b94f24be338f86`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:32:49.010438 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-957d99bb-4ded-4c0d-a0e8-4b09237c0501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:49.074273 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1394/2762] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Jul 27 04:32:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:32:49.122156 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6ce30-ad5b-4ca7-98df-de69e5a1560f service nova] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:32:49.123563 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-79f6ce30-ad5b-4ca7-98df-de69e5a1560f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:32:49.216417 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6ce30-ad5b-4ca7-98df-de69e5a1560f service nova] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:32:49.221248 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414b637-d89d-46a0-b8cd-aff3ce294d28 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57b5fa7517034a39930ce7c9d12e1875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:32:49.221858 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414b637-d89d-46a0-b8cd-aff3ce294d28 service swift] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:32:49.222168 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414b637-d89d-46a0-b8cd-aff3ce294d28 service swift] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:32:49.222444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414b637-d89d-46a0-b8cd-aff3ce294d28 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:32:49.230350 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414b637-d89d-46a0-b8cd-aff3ce294d28 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:32:49.231916 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414b637-d89d-46a0-b8cd-aff3ce294d28 service swift] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:32:49.239688 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1369/2763] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 27 04:32:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 04:32:59.071122 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=44 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:32:59.236686 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=47 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:33:03.824440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2e84e-93e1-4cf2-83ba-a407c3d82cb3 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:03.824935 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2e84e-93e1-4cf2-83ba-a407c3d82cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:03.825039 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2e84e-93e1-4cf2-83ba-a407c3d82cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:03.843537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-1ae2e84e-93e1-4cf2-83ba-a407c3d82cb3 None None] MFA Rules not processed for user `8ec07ee1efba4e8089b94f24be338f86`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:33:03.844724 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-1ae2e84e-93e1-4cf2-83ba-a407c3d82cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:03.852243 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1395/2764] 10.222.0.46 () {58 vars in 967 bytes} [Wed Jul 27 04:33:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:33:03.863809 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414b637-d89d-46a0-b8cd-aff3ce294d28 service swift] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:03.875595 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414b637-d89d-46a0-b8cd-aff3ce294d28 service swift] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:03.880808 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ced214-663d-4aa3-ab26-5bbd7caf01bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57b5fa7517034a39930ce7c9d12e1875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:03.881558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ced214-663d-4aa3-ab26-5bbd7caf01bd service swift] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:03.881912 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ced214-663d-4aa3-ab26-5bbd7caf01bd service swift] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:03.882287 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ced214-663d-4aa3-ab26-5bbd7caf01bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:03.893468 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ced214-663d-4aa3-ab26-5bbd7caf01bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:03.895337 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ced214-663d-4aa3-ab26-5bbd7caf01bd service swift] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:03.903856 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1370/2765] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 27 04:33:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 04:33:05.732775 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2e84e-93e1-4cf2-83ba-a407c3d82cb3 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:05.745734 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2e84e-93e1-4cf2-83ba-a407c3d82cb3 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:05.750752 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37b1f1-c430-4dbe-9024-b4ae9d2811d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c08fba2d7de42218400c5e652cc3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:05.751373 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37b1f1-c430-4dbe-9024-b4ae9d2811d6 service placement] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:05.751658 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37b1f1-c430-4dbe-9024-b4ae9d2811d6 service placement] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:05.751964 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37b1f1-c430-4dbe-9024-b4ae9d2811d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:05.753721 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-dc37b1f1-c430-4dbe-9024-b4ae9d2811d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:05.766160 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc37b1f1-c430-4dbe-9024-b4ae9d2811d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:05.767842 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc37b1f1-c430-4dbe-9024-b4ae9d2811d6 service placement] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:05.778582 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1396/2766] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:33:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:33:05.798030 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ced214-663d-4aa3-ab26-5bbd7caf01bd service swift] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:05.811237 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ced214-663d-4aa3-ab26-5bbd7caf01bd service swift] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:05.816104 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe053fba-bec1-44c2-a811-5906690cff06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:05.816814 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe053fba-bec1-44c2-a811-5906690cff06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:05.817388 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe053fba-bec1-44c2-a811-5906690cff06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:05.817672 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe053fba-bec1-44c2-a811-5906690cff06 admin admin] PATH_INFO: `/v3/users/69e9a87bc39f4456ab90586e02d0785d` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:05.819562 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe053fba-bec1-44c2-a811-5906690cff06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69e9a87bc39f4456ab90586e02d0785d)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:05.821532 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe053fba-bec1-44c2-a811-5906690cff06 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:05.868850 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-fe053fba-bec1-44c2-a811-5906690cff06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69e9a87bc39f4456ab90586e02d0785d'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:05.869503 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-fe053fba-bec1-44c2-a811-5906690cff06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69e9a87bc39f4456ab90586e02d0785d'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:05.877208 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-fe053fba-bec1-44c2-a811-5906690cff06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69e9a87bc39f4456ab90586e02d0785d'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:05.889805 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-fe053fba-bec1-44c2-a811-5906690cff06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69e9a87bc39f4456ab90586e02d0785d'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:05.896558 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-fe053fba-bec1-44c2-a811-5906690cff06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69e9a87bc39f4456ab90586e02d0785d'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:05.902736 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1371/2767] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:33:05 2022] DELETE /identity/v3/users/69e9a87bc39f4456ab90586e02d0785d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:33:06.076285 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37b1f1-c430-4dbe-9024-b4ae9d2811d6 service placement] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:06.087407 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37b1f1-c430-4dbe-9024-b4ae9d2811d6 service placement] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:06.148327 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67dc3a-fafa-463b-a01e-e93871767403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:06.148963 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67dc3a-fafa-463b-a01e-e93871767403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:06.149247 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67dc3a-fafa-463b-a01e-e93871767403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:06.149520 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67dc3a-fafa-463b-a01e-e93871767403 admin admin] PATH_INFO: `/v3/projects/37f8c051d2c14448ab9a73672f3f00a9` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:06.150746 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa67dc3a-fafa-463b-a01e-e93871767403 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37f8c051d2c14448ab9a73672f3f00a9)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:06.153073 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa67dc3a-fafa-463b-a01e-e93871767403 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:06.201699 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fa67dc3a-fafa-463b-a01e-e93871767403 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37f8c051d2c14448ab9a73672f3f00a9', 'request_id': 'req-fa67dc3a-fafa-463b-a01e-e93871767403'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:06.207851 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fa67dc3a-fafa-463b-a01e-e93871767403 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37f8c051d2c14448ab9a73672f3f00a9', 'request_id': 'req-fa67dc3a-fafa-463b-a01e-e93871767403'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:06.213195 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fa67dc3a-fafa-463b-a01e-e93871767403 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37f8c051d2c14448ab9a73672f3f00a9', 'request_id': 'req-fa67dc3a-fafa-463b-a01e-e93871767403'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:06.225911 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fa67dc3a-fafa-463b-a01e-e93871767403 admin admin] The token cache is being invalidate because project 37f8c051d2c14448ab9a73672f3f00a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:33:06.226232 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fa67dc3a-fafa-463b-a01e-e93871767403 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:06.226864 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fa67dc3a-fafa-463b-a01e-e93871767403 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:06.228446 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1397/2768] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:33:06 2022] DELETE /identity/v3/projects/37f8c051d2c14448ab9a73672f3f00a9 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:33:06.233444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36870998-7d25-4e7a-8c6f-6192dfdcbabb None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:06.233736 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36870998-7d25-4e7a-8c6f-6192dfdcbabb None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:06.234049 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36870998-7d25-4e7a-8c6f-6192dfdcbabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:06.254243 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-36870998-7d25-4e7a-8c6f-6192dfdcbabb None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:33:06.255444 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-36870998-7d25-4e7a-8c6f-6192dfdcbabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:06.314350 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1372/2769] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:33:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:33:06.321820 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c22282-f756-4ae7-b34c-7162a5ac6e96 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:06.322134 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c22282-f756-4ae7-b34c-7162a5ac6e96 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:06.322412 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c22282-f756-4ae7-b34c-7162a5ac6e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:06.343395 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-b2c22282-f756-4ae7-b34c-7162a5ac6e96 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:33:06.344584 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b2c22282-f756-4ae7-b34c-7162a5ac6e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:06.353096 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1398/2770] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:33:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:33:06.357189 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36870998-7d25-4e7a-8c6f-6192dfdcbabb None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:06.366945 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36870998-7d25-4e7a-8c6f-6192dfdcbabb None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:06.371424 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518d6e8-703d-45d3-a04a-118ab0c9a243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:06.372029 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518d6e8-703d-45d3-a04a-118ab0c9a243 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:06.372309 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518d6e8-703d-45d3-a04a-118ab0c9a243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:06.372578 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518d6e8-703d-45d3-a04a-118ab0c9a243 admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:06.373465 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2518d6e8-703d-45d3-a04a-118ab0c9a243 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:33:06.373767 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2518d6e8-703d-45d3-a04a-118ab0c9a243 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:06.375383 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2518d6e8-703d-45d3-a04a-118ab0c9a243 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:06.386741 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1373/2771] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:33:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:33:06.390850 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c22282-f756-4ae7-b34c-7162a5ac6e96 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:06.400760 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c22282-f756-4ae7-b34c-7162a5ac6e96 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:06.405285 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bcc273-3910-41e9-92d5-361c25532441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:06.405914 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bcc273-3910-41e9-92d5-361c25532441 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:06.406224 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bcc273-3910-41e9-92d5-361c25532441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:06.406500 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bcc273-3910-41e9-92d5-361c25532441 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:06.407175 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bcc273-3910-41e9-92d5-361c25532441 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:06.409055 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bcc273-3910-41e9-92d5-361c25532441 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:06.435548 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1399/2772] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:33:06 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:33:06.439478 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518d6e8-703d-45d3-a04a-118ab0c9a243 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:06.449454 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518d6e8-703d-45d3-a04a-118ab0c9a243 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:06.493291 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:33:06.507999 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499043f8-e08a-4f01-bc66-5b1489bc2731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:06.508626 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499043f8-e08a-4f01-bc66-5b1489bc2731 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:06.508904 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499043f8-e08a-4f01-bc66-5b1489bc2731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:06.509175 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499043f8-e08a-4f01-bc66-5b1489bc2731 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:06.509836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499043f8-e08a-4f01-bc66-5b1489bc2731 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:06.512912 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499043f8-e08a-4f01-bc66-5b1489bc2731 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:06.539499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1374/2773] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:33:06 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:33:06.543718 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bcc273-3910-41e9-92d5-361c25532441 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:06.554062 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bcc273-3910-41e9-92d5-361c25532441 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:06.558666 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe8fc4-48c7-4193-9948-b742b6fa6797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:06.559269 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe8fc4-48c7-4193-9948-b742b6fa6797 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:06.559550 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe8fc4-48c7-4193-9948-b742b6fa6797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:06.559821 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe8fc4-48c7-4193-9948-b742b6fa6797 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:06.560513 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe8fc4-48c7-4193-9948-b742b6fa6797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:06.562152 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe8fc4-48c7-4193-9948-b742b6fa6797 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:06.571959 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1400/2774] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:33:06.576038 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499043f8-e08a-4f01-bc66-5b1489bc2731 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:06.586389 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499043f8-e08a-4f01-bc66-5b1489bc2731 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:06.591007 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2770dba8-c3fc-430a-a9ed-e7d0057f2b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:06.591609 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2770dba8-c3fc-430a-a9ed-e7d0057f2b58 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:06.591894 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2770dba8-c3fc-430a-a9ed-e7d0057f2b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:06.592173 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2770dba8-c3fc-430a-a9ed-e7d0057f2b58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:06.592871 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2770dba8-c3fc-430a-a9ed-e7d0057f2b58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:06.594489 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2770dba8-c3fc-430a-a9ed-e7d0057f2b58 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:06.608724 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1375/2775] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:33:06.613398 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe8fc4-48c7-4193-9948-b742b6fa6797 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:06.623521 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe8fc4-48c7-4193-9948-b742b6fa6797 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:06.628186 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef35ff-a3bf-4102-bd58-7f5460b1c2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:06.628843 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef35ff-a3bf-4102-bd58-7f5460b1c2d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:06.629126 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef35ff-a3bf-4102-bd58-7f5460b1c2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:06.629403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef35ff-a3bf-4102-bd58-7f5460b1c2d2 admin admin] PATH_INFO: `/v3/projects/2a7c0601e0fa4365a8d06926cd3faf23/users/a0a8ef82c8c9420fbe209fe3bd26efac/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:06.644509 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef35ff-a3bf-4102-bd58-7f5460b1c2d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a7c0601e0fa4365a8d06926cd3faf23, user_id=a0a8ef82c8c9420fbe209fe3bd26efac, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:06.647915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef35ff-a3bf-4102-bd58-7f5460b1c2d2 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:06.660606 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1401/2776] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:33:06 2022] PUT /identity/v3/projects/2a7c0601e0fa4365a8d06926cd3faf23/users/a0a8ef82c8c9420fbe209fe3bd26efac/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:33:06.671388 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce79b3-d79a-4a73-9bf3-b79f95a72a99 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:06.671674 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce79b3-d79a-4a73-9bf3-b79f95a72a99 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:06.671949 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce79b3-d79a-4a73-9bf3-b79f95a72a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:06.699447 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-e4ce79b3-d79a-4a73-9bf3-b79f95a72a99 None None] MFA Rules not processed for user `a0a8ef82c8c9420fbe209fe3bd26efac`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:33:06.700741 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e4ce79b3-d79a-4a73-9bf3-b79f95a72a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:06.763840 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1376/2777] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:33:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:33:06.774905 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef35ff-a3bf-4102-bd58-7f5460b1c2d2 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:06.776617 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5cef35ff-a3bf-4102-bd58-7f5460b1c2d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:06.786168 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:33:06.837651 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef35ff-a3bf-4102-bd58-7f5460b1c2d2 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:06.841254 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b896c9-0237-4c48-a13d-490d0dd1a97e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:06.841865 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b896c9-0237-4c48-a13d-490d0dd1a97e service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:06.842179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b896c9-0237-4c48-a13d-490d0dd1a97e service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:06.842451 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b896c9-0237-4c48-a13d-490d0dd1a97e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:06.850751 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b896c9-0237-4c48-a13d-490d0dd1a97e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:06.852162 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b896c9-0237-4c48-a13d-490d0dd1a97e service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:06.860793 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1402/2778] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:33:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 04:33:16.761154 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=45,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:33:16.824276 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce79b3-d79a-4a73-9bf3-b79f95a72a99 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:16.856808 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=23 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:33:16.886731 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce79b3-d79a-4a73-9bf3-b79f95a72a99 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:16.891171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6a500b-4a9d-4c2e-a539-b7281b09778c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:16.891802 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a500b-4a9d-4c2e-a539-b7281b09778c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:16.892079 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a500b-4a9d-4c2e-a539-b7281b09778c admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:16.892349 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a500b-4a9d-4c2e-a539-b7281b09778c admin admin] PATH_INFO: `/v3/users/a0a8ef82c8c9420fbe209fe3bd26efac` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:16.893484 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6a500b-4a9d-4c2e-a539-b7281b09778c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0a8ef82c8c9420fbe209fe3bd26efac)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:16.895420 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6a500b-4a9d-4c2e-a539-b7281b09778c admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:16.941193 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2f6a500b-4a9d-4c2e-a539-b7281b09778c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0a8ef82c8c9420fbe209fe3bd26efac'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:16.941936 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2f6a500b-4a9d-4c2e-a539-b7281b09778c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0a8ef82c8c9420fbe209fe3bd26efac'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:16.949590 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2f6a500b-4a9d-4c2e-a539-b7281b09778c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0a8ef82c8c9420fbe209fe3bd26efac'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:16.961295 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2f6a500b-4a9d-4c2e-a539-b7281b09778c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0a8ef82c8c9420fbe209fe3bd26efac'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:16.968615 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2f6a500b-4a9d-4c2e-a539-b7281b09778c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0a8ef82c8c9420fbe209fe3bd26efac'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:16.974421 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1377/2779] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:33:16 2022] DELETE /identity/v3/users/a0a8ef82c8c9420fbe209fe3bd26efac => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:33:16.978837 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fdab20-e89c-41f4-88c3-5a8e50e447ac None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:16.979779 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fdab20-e89c-41f4-88c3-5a8e50e447ac None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:16.980091 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fdab20-e89c-41f4-88c3-5a8e50e447ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:16.998355 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-d1fdab20-e89c-41f4-88c3-5a8e50e447ac None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:33:16.999537 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d1fdab20-e89c-41f4-88c3-5a8e50e447ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:17.056699 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1403/2780] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:33:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:33:17.066318 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6a500b-4a9d-4c2e-a539-b7281b09778c admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:17.067671 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2f6a500b-4a9d-4c2e-a539-b7281b09778c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:17.123937 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6a500b-4a9d-4c2e-a539-b7281b09778c admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:17.127521 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb09121-0ad4-4faa-95d7-d900d0dda4b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:17.128131 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb09121-0ad4-4faa-95d7-d900d0dda4b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:17.128410 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb09121-0ad4-4faa-95d7-d900d0dda4b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:17.128683 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb09121-0ad4-4faa-95d7-d900d0dda4b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:17.138372 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb09121-0ad4-4faa-95d7-d900d0dda4b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:17.139804 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb09121-0ad4-4faa-95d7-d900d0dda4b5 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:17.149283 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1378/2781] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:33:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:33:17.418343 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fdab20-e89c-41f4-88c3-5a8e50e447ac None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:17.419927 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d1fdab20-e89c-41f4-88c3-5a8e50e447ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:17.433606 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fdab20-e89c-41f4-88c3-5a8e50e447ac None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:17.438631 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c7f7a-8ae2-4959-98f1-1198739f74a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:17.439287 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c7f7a-8ae2-4959-98f1-1198739f74a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:17.439565 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c7f7a-8ae2-4959-98f1-1198739f74a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:17.439835 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c7f7a-8ae2-4959-98f1-1198739f74a5 admin admin] PATH_INFO: `/v3/projects/2a7c0601e0fa4365a8d06926cd3faf23` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:17.441145 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162c7f7a-8ae2-4959-98f1-1198739f74a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a7c0601e0fa4365a8d06926cd3faf23)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:17.443213 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162c7f7a-8ae2-4959-98f1-1198739f74a5 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:17.491310 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-162c7f7a-8ae2-4959-98f1-1198739f74a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a7c0601e0fa4365a8d06926cd3faf23', 'request_id': 'req-162c7f7a-8ae2-4959-98f1-1198739f74a5'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:17.497852 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-162c7f7a-8ae2-4959-98f1-1198739f74a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a7c0601e0fa4365a8d06926cd3faf23', 'request_id': 'req-162c7f7a-8ae2-4959-98f1-1198739f74a5'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:17.503417 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-162c7f7a-8ae2-4959-98f1-1198739f74a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a7c0601e0fa4365a8d06926cd3faf23', 'request_id': 'req-162c7f7a-8ae2-4959-98f1-1198739f74a5'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:17.515883 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-162c7f7a-8ae2-4959-98f1-1198739f74a5 admin admin] The token cache is being invalidate because project 2a7c0601e0fa4365a8d06926cd3faf23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:33:17.516174 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-162c7f7a-8ae2-4959-98f1-1198739f74a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:17.516807 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-162c7f7a-8ae2-4959-98f1-1198739f74a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:17.518362 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1404/2782] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:33:17 2022] DELETE /identity/v3/projects/2a7c0601e0fa4365a8d06926cd3faf23 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:33:17.523421 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00edfa-b8d6-4c1e-a6d0-50052b5e15d7 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:17.523711 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00edfa-b8d6-4c1e-a6d0-50052b5e15d7 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:17.523989 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00edfa-b8d6-4c1e-a6d0-50052b5e15d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:17.542959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-ec00edfa-b8d6-4c1e-a6d0-50052b5e15d7 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:33:17.544154 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ec00edfa-b8d6-4c1e-a6d0-50052b5e15d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:17.560947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:33:17.606086 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1379/2783] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:33:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:33:17.613770 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd46db-f692-41d4-a5aa-c695c71fd705 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:17.614099 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd46db-f692-41d4-a5aa-c695c71fd705 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:17.614377 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd46db-f692-41d4-a5aa-c695c71fd705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:17.634228 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-c4fd46db-f692-41d4-a5aa-c695c71fd705 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:33:17.636112 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c4fd46db-f692-41d4-a5aa-c695c71fd705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:17.645584 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1405/2784] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:33:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:33:17.650034 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec00edfa-b8d6-4c1e-a6d0-50052b5e15d7 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:17.660027 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec00edfa-b8d6-4c1e-a6d0-50052b5e15d7 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:17.664686 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f7979-3202-47fc-9f5b-b6fae390cb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:17.665305 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f7979-3202-47fc-9f5b-b6fae390cb61 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:17.665588 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f7979-3202-47fc-9f5b-b6fae390cb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:17.665864 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f7979-3202-47fc-9f5b-b6fae390cb61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:17.667267 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210f7979-3202-47fc-9f5b-b6fae390cb61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:33:17.667751 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210f7979-3202-47fc-9f5b-b6fae390cb61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:17.669354 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210f7979-3202-47fc-9f5b-b6fae390cb61 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:17.681538 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1380/2785] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:33:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:33:17.686784 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd46db-f692-41d4-a5aa-c695c71fd705 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:17.699518 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd46db-f692-41d4-a5aa-c695c71fd705 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:17.704696 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa3ac4-8036-41cf-ba60-33fb9adfd3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:17.705535 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa3ac4-8036-41cf-ba60-33fb9adfd3ea admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:17.706109 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa3ac4-8036-41cf-ba60-33fb9adfd3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:17.706544 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa3ac4-8036-41cf-ba60-33fb9adfd3ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:17.707500 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa3ac4-8036-41cf-ba60-33fb9adfd3ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:17.709584 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa3ac4-8036-41cf-ba60-33fb9adfd3ea admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:17.744233 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1406/2786] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:33:17 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:33:17.748696 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f7979-3202-47fc-9f5b-b6fae390cb61 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:17.758780 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f7979-3202-47fc-9f5b-b6fae390cb61 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:17.818905 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6dff9-8945-4539-bb1a-039ffea59b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:17.819612 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6dff9-8945-4539-bb1a-039ffea59b88 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:17.819816 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6dff9-8945-4539-bb1a-039ffea59b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:17.820087 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6dff9-8945-4539-bb1a-039ffea59b88 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:17.820774 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c6dff9-8945-4539-bb1a-039ffea59b88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:17.823346 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c6dff9-8945-4539-bb1a-039ffea59b88 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:17.849805 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1381/2787] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:33:17 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:33:17.851065 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa3ac4-8036-41cf-ba60-33fb9adfd3ea admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:17.861917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa3ac4-8036-41cf-ba60-33fb9adfd3ea admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:17.866448 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d0bd1-30c2-4112-a011-247d791bdc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:17.867154 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d0bd1-30c2-4112-a011-247d791bdc54 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:17.867484 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d0bd1-30c2-4112-a011-247d791bdc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:17.867824 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d0bd1-30c2-4112-a011-247d791bdc54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:17.870446 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d0bd1-30c2-4112-a011-247d791bdc54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:17.872065 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d0bd1-30c2-4112-a011-247d791bdc54 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:17.882416 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1407/2788] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:33:17.887100 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6dff9-8945-4539-bb1a-039ffea59b88 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:17.897655 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6dff9-8945-4539-bb1a-039ffea59b88 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:17.903039 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c610ffc-8ffa-4003-8fab-2846aa90afcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:17.903720 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c610ffc-8ffa-4003-8fab-2846aa90afcc admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:17.904139 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c610ffc-8ffa-4003-8fab-2846aa90afcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:17.904261 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c610ffc-8ffa-4003-8fab-2846aa90afcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:17.904918 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c610ffc-8ffa-4003-8fab-2846aa90afcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:17.906620 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c610ffc-8ffa-4003-8fab-2846aa90afcc admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:17.919603 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1382/2789] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 04:33:17.923820 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d0bd1-30c2-4112-a011-247d791bdc54 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:17.934732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d0bd1-30c2-4112-a011-247d791bdc54 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:17.939379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d3870-98b0-4f32-982a-a13f0fbaee4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:17.940039 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d3870-98b0-4f32-982a-a13f0fbaee4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:17.940321 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d3870-98b0-4f32-982a-a13f0fbaee4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:17.940601 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d3870-98b0-4f32-982a-a13f0fbaee4e admin admin] PATH_INFO: `/v3/projects/4f9d86602f8f4430a42e9d5e829c0cef/users/6f7abba95ba04197b6af7e5e20bbc824/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:17.955934 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d3870-98b0-4f32-982a-a13f0fbaee4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f9d86602f8f4430a42e9d5e829c0cef, user_id=6f7abba95ba04197b6af7e5e20bbc824, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:17.958612 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d3870-98b0-4f32-982a-a13f0fbaee4e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:17.971736 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1408/2790] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:33:17 2022] PUT /identity/v3/projects/4f9d86602f8f4430a42e9d5e829c0cef/users/6f7abba95ba04197b6af7e5e20bbc824/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:33:17.983582 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba07e86-ca60-4bd8-8b08-e368ebceef82 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:17.983863 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba07e86-ca60-4bd8-8b08-e368ebceef82 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:17.984136 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba07e86-ca60-4bd8-8b08-e368ebceef82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:18.012806 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-fba07e86-ca60-4bd8-8b08-e368ebceef82 None None] MFA Rules not processed for user `6f7abba95ba04197b6af7e5e20bbc824`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:33:18.014100 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-fba07e86-ca60-4bd8-8b08-e368ebceef82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:18.077272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1383/2791] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:33:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:33:18.088078 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d3870-98b0-4f32-982a-a13f0fbaee4e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:18.089848 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-269d3870-98b0-4f32-982a-a13f0fbaee4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:18.111443 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:33:18.148692 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d3870-98b0-4f32-982a-a13f0fbaee4e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:18.152765 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2766fde0-2045-4b25-8d01-eaa25e0b8dcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:18.153379 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766fde0-2045-4b25-8d01-eaa25e0b8dcd service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:18.153658 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766fde0-2045-4b25-8d01-eaa25e0b8dcd service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:18.153938 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766fde0-2045-4b25-8d01-eaa25e0b8dcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:18.162100 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2766fde0-2045-4b25-8d01-eaa25e0b8dcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:18.163529 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2766fde0-2045-4b25-8d01-eaa25e0b8dcd service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:18.172198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1409/2792] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:33:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:33:19.524991 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde2f90-5e1c-44bc-94d1-4cd4e394cd89 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:19.525473 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde2f90-5e1c-44bc-94d1-4cd4e394cd89 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:19.525597 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde2f90-5e1c-44bc-94d1-4cd4e394cd89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:19.544637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-edde2f90-5e1c-44bc-94d1-4cd4e394cd89 None None] MFA Rules not processed for user `6f7abba95ba04197b6af7e5e20bbc824`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:33:19.545842 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-edde2f90-5e1c-44bc-94d1-4cd4e394cd89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:19.552725 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1384/2793] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:33:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:33:28.168334 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=17 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:33:29.538856 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:33:33.248198 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5fb07-1fea-475d-a433-e436145cb852 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:33.248881 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5fb07-1fea-475d-a433-e436145cb852 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:33.248982 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5fb07-1fea-475d-a433-e436145cb852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:33.270600 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-1db5fb07-1fea-475d-a433-e436145cb852 None None] MFA Rules not processed for user `8ec07ee1efba4e8089b94f24be338f86`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:33:33.272023 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-1db5fb07-1fea-475d-a433-e436145cb852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:33.340834 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1410/2794] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Jul 27 04:33:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:33:33.352591 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde2f90-5e1c-44bc-94d1-4cd4e394cd89 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:33.354074 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-edde2f90-5e1c-44bc-94d1-4cd4e394cd89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:33.427168 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde2f90-5e1c-44bc-94d1-4cd4e394cd89 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:33.432114 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c032a2-c5aa-4ada-bdbe-64b015ea1f32 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57b5fa7517034a39930ce7c9d12e1875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:33.433297 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c032a2-c5aa-4ada-bdbe-64b015ea1f32 service swift] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:33.433761 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c032a2-c5aa-4ada-bdbe-64b015ea1f32 service swift] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:33.434113 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c032a2-c5aa-4ada-bdbe-64b015ea1f32 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:33.442304 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c032a2-c5aa-4ada-bdbe-64b015ea1f32 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:33.443895 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c032a2-c5aa-4ada-bdbe-64b015ea1f32 service swift] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:33.453253 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1385/2795] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 27 04:33:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 04:33:33.958726 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c348f3-515f-43da-911f-522c9f141e52 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:33.959017 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c348f3-515f-43da-911f-522c9f141e52 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:33.959297 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c348f3-515f-43da-911f-522c9f141e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:33.978580 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-91c348f3-515f-43da-911f-522c9f141e52 None None] MFA Rules not processed for user `8ec07ee1efba4e8089b94f24be338f86`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:33:33.980368 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-91c348f3-515f-43da-911f-522c9f141e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:33.987425 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1411/2796] 10.222.0.46 () {58 vars in 967 bytes} [Wed Jul 27 04:33:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:33:34.000033 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c032a2-c5aa-4ada-bdbe-64b015ea1f32 service swift] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:34.010399 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c032a2-c5aa-4ada-bdbe-64b015ea1f32 service swift] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:34.015637 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a381606f-bdde-4f84-bb0f-4a122544fb0c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57b5fa7517034a39930ce7c9d12e1875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:34.016232 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381606f-bdde-4f84-bb0f-4a122544fb0c service swift] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:34.016510 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381606f-bdde-4f84-bb0f-4a122544fb0c service swift] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:34.016779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381606f-bdde-4f84-bb0f-4a122544fb0c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:34.024609 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a381606f-bdde-4f84-bb0f-4a122544fb0c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:34.026933 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a381606f-bdde-4f84-bb0f-4a122544fb0c service swift] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:34.034800 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1386/2797] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 27 04:33:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 04:33:34.244032 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548e647-bef5-4128-bdc1-5a6c261fab5f None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:34.244316 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548e647-bef5-4128-bdc1-5a6c261fab5f None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:34.244589 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548e647-bef5-4128-bdc1-5a6c261fab5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:34.263440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-2548e647-bef5-4128-bdc1-5a6c261fab5f None None] MFA Rules not processed for user `8ec07ee1efba4e8089b94f24be338f86`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:33:34.264608 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2548e647-bef5-4128-bdc1-5a6c261fab5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:34.271660 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1412/2798] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Jul 27 04:33:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:33:34.284125 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a381606f-bdde-4f84-bb0f-4a122544fb0c service swift] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:34.294667 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a381606f-bdde-4f84-bb0f-4a122544fb0c service swift] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:34.299660 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8207bf16-886b-4fbe-a4f3-f35cec46bdba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57b5fa7517034a39930ce7c9d12e1875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:34.300261 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207bf16-886b-4fbe-a4f3-f35cec46bdba service swift] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:34.300542 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207bf16-886b-4fbe-a4f3-f35cec46bdba service swift] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:34.300813 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207bf16-886b-4fbe-a4f3-f35cec46bdba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:34.308851 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8207bf16-886b-4fbe-a4f3-f35cec46bdba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:34.310455 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8207bf16-886b-4fbe-a4f3-f35cec46bdba service swift] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:34.318407 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1387/2799] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 27 04:33:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 04:33:34.637720 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c43eb-c3d9-4a9d-9800-12810aefd866 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:34.638042 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c43eb-c3d9-4a9d-9800-12810aefd866 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:34.638325 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c43eb-c3d9-4a9d-9800-12810aefd866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:34.657210 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-542c43eb-c3d9-4a9d-9800-12810aefd866 None None] MFA Rules not processed for user `8ec07ee1efba4e8089b94f24be338f86`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:33:34.658422 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-542c43eb-c3d9-4a9d-9800-12810aefd866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:34.665405 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1413/2800] 10.222.0.46 () {58 vars in 967 bytes} [Wed Jul 27 04:33:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:33:34.677659 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8207bf16-886b-4fbe-a4f3-f35cec46bdba service swift] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:34.689147 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8207bf16-886b-4fbe-a4f3-f35cec46bdba service swift] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:34.694398 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a272a3-e94a-4d26-87e0-eba7afd7765d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57b5fa7517034a39930ce7c9d12e1875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:34.694946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a272a3-e94a-4d26-87e0-eba7afd7765d service swift] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:34.695225 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a272a3-e94a-4d26-87e0-eba7afd7765d service swift] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:34.695498 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a272a3-e94a-4d26-87e0-eba7afd7765d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:34.704324 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a272a3-e94a-4d26-87e0-eba7afd7765d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:34.705892 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a272a3-e94a-4d26-87e0-eba7afd7765d service swift] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:34.713799 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1388/2801] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 27 04:33:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 04:33:36.789681 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c43eb-c3d9-4a9d-9800-12810aefd866 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:36.851602 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c43eb-c3d9-4a9d-9800-12810aefd866 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:36.855988 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce72fa-76f9-4b74-8b92-2f932261c901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:36.856615 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce72fa-76f9-4b74-8b92-2f932261c901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:36.856894 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce72fa-76f9-4b74-8b92-2f932261c901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:36.857166 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce72fa-76f9-4b74-8b92-2f932261c901 admin admin] PATH_INFO: `/v3/users/6f7abba95ba04197b6af7e5e20bbc824` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:36.858353 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce72fa-76f9-4b74-8b92-2f932261c901 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f7abba95ba04197b6af7e5e20bbc824)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:36.860254 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce72fa-76f9-4b74-8b92-2f932261c901 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:36.910559 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-acce72fa-76f9-4b74-8b92-2f932261c901 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f7abba95ba04197b6af7e5e20bbc824'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:36.911203 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-acce72fa-76f9-4b74-8b92-2f932261c901 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f7abba95ba04197b6af7e5e20bbc824'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:36.911817 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-acce72fa-76f9-4b74-8b92-2f932261c901 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f7abba95ba04197b6af7e5e20bbc824'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:36.924461 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-acce72fa-76f9-4b74-8b92-2f932261c901 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f7abba95ba04197b6af7e5e20bbc824'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:36.931244 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-acce72fa-76f9-4b74-8b92-2f932261c901 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f7abba95ba04197b6af7e5e20bbc824'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:36.948556 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1414/2802] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:33:36 2022] DELETE /identity/v3/users/6f7abba95ba04197b6af7e5e20bbc824 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:33:36.953363 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6cb0e-9364-4f0d-9117-42c35d63be13 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:36.953864 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6cb0e-9364-4f0d-9117-42c35d63be13 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:36.954014 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6cb0e-9364-4f0d-9117-42c35d63be13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:36.975817 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-10d6cb0e-9364-4f0d-9117-42c35d63be13 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:33:36.977088 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-10d6cb0e-9364-4f0d-9117-42c35d63be13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:37.035515 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1389/2803] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:33:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:33:37.045842 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce72fa-76f9-4b74-8b92-2f932261c901 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:37.047364 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-acce72fa-76f9-4b74-8b92-2f932261c901 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:37.053856 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:33:37.106589 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce72fa-76f9-4b74-8b92-2f932261c901 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:37.110419 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd67ee-2089-4b83-9350-92f035df173e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:37.111031 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd67ee-2089-4b83-9350-92f035df173e service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:37.111312 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd67ee-2089-4b83-9350-92f035df173e service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:37.111576 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd67ee-2089-4b83-9350-92f035df173e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:37.121322 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bd67ee-2089-4b83-9350-92f035df173e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:37.122812 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bd67ee-2089-4b83-9350-92f035df173e service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:37.132093 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1415/2804] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:33:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 04:33:37.303572 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6cb0e-9364-4f0d-9117-42c35d63be13 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:37.304985 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-10d6cb0e-9364-4f0d-9117-42c35d63be13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:37.321160 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6cb0e-9364-4f0d-9117-42c35d63be13 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:37.326248 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff275f-2606-44df-a6b2-ff97b2bd9d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:37.326907 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff275f-2606-44df-a6b2-ff97b2bd9d82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:37.327187 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff275f-2606-44df-a6b2-ff97b2bd9d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:37.327464 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff275f-2606-44df-a6b2-ff97b2bd9d82 admin admin] PATH_INFO: `/v3/projects/4f9d86602f8f4430a42e9d5e829c0cef` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:37.328821 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cff275f-2606-44df-a6b2-ff97b2bd9d82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f9d86602f8f4430a42e9d5e829c0cef)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:37.330814 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cff275f-2606-44df-a6b2-ff97b2bd9d82 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:37.376145 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:33:37.379361 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2cff275f-2606-44df-a6b2-ff97b2bd9d82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f9d86602f8f4430a42e9d5e829c0cef', 'request_id': 'req-2cff275f-2606-44df-a6b2-ff97b2bd9d82'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:37.385987 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2cff275f-2606-44df-a6b2-ff97b2bd9d82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f9d86602f8f4430a42e9d5e829c0cef', 'request_id': 'req-2cff275f-2606-44df-a6b2-ff97b2bd9d82'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:37.391373 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2cff275f-2606-44df-a6b2-ff97b2bd9d82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f9d86602f8f4430a42e9d5e829c0cef', 'request_id': 'req-2cff275f-2606-44df-a6b2-ff97b2bd9d82'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:37.403131 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2cff275f-2606-44df-a6b2-ff97b2bd9d82 admin admin] The token cache is being invalidate because project 4f9d86602f8f4430a42e9d5e829c0cef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:33:37.403406 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2cff275f-2606-44df-a6b2-ff97b2bd9d82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:37.404019 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-2cff275f-2606-44df-a6b2-ff97b2bd9d82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:37.405499 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1390/2805] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:33:37 2022] DELETE /identity/v3/projects/4f9d86602f8f4430a42e9d5e829c0cef => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:33:47.127905 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=24 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:33:47.400179 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:33:53.795985 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd67ee-2089-4b83-9350-92f035df173e service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:53.797242 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-94bd67ee-2089-4b83-9350-92f035df173e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:53.811565 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd67ee-2089-4b83-9350-92f035df173e service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:53.871460 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d57f0-5817-48be-9e67-41de8b82005f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:53.872294 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d57f0-5817-48be-9e67-41de8b82005f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:53.872575 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d57f0-5817-48be-9e67-41de8b82005f admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:53.872855 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d57f0-5817-48be-9e67-41de8b82005f admin admin] PATH_INFO: `/v3/users/89aa18ddc71c4951bf15fcbd404912e9` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:53.874156 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d57f0-5817-48be-9e67-41de8b82005f admin admin] RBAC: Authorizing `identity:delete_user(user_id=89aa18ddc71c4951bf15fcbd404912e9)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:53.876094 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d57f0-5817-48be-9e67-41de8b82005f admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:53.927335 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fb7d57f0-5817-48be-9e67-41de8b82005f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89aa18ddc71c4951bf15fcbd404912e9'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:53.928360 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fb7d57f0-5817-48be-9e67-41de8b82005f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89aa18ddc71c4951bf15fcbd404912e9'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:53.928805 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fb7d57f0-5817-48be-9e67-41de8b82005f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89aa18ddc71c4951bf15fcbd404912e9'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:53.943208 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fb7d57f0-5817-48be-9e67-41de8b82005f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89aa18ddc71c4951bf15fcbd404912e9'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:53.950920 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fb7d57f0-5817-48be-9e67-41de8b82005f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89aa18ddc71c4951bf15fcbd404912e9'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:53.965212 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1416/2806] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:33:53 2022] DELETE /identity/v3/users/89aa18ddc71c4951bf15fcbd404912e9 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:33:54.139643 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff275f-2606-44df-a6b2-ff97b2bd9d82 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:54.143106 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-2cff275f-2606-44df-a6b2-ff97b2bd9d82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:54.158686 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff275f-2606-44df-a6b2-ff97b2bd9d82 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:54.215520 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffd47e-2355-4a3a-8e46-ee7f64807043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:54.216621 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffd47e-2355-4a3a-8e46-ee7f64807043 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:54.216897 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffd47e-2355-4a3a-8e46-ee7f64807043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:54.217163 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffd47e-2355-4a3a-8e46-ee7f64807043 admin admin] PATH_INFO: `/v3/projects/e51f9e52588e4008adf20f443c367b02` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:54.218353 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebffd47e-2355-4a3a-8e46-ee7f64807043 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e51f9e52588e4008adf20f443c367b02)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:54.220254 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebffd47e-2355-4a3a-8e46-ee7f64807043 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:54.267535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ebffd47e-2355-4a3a-8e46-ee7f64807043 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e51f9e52588e4008adf20f443c367b02', 'request_id': 'req-ebffd47e-2355-4a3a-8e46-ee7f64807043'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:54.273516 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ebffd47e-2355-4a3a-8e46-ee7f64807043 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e51f9e52588e4008adf20f443c367b02', 'request_id': 'req-ebffd47e-2355-4a3a-8e46-ee7f64807043'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:54.282675 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ebffd47e-2355-4a3a-8e46-ee7f64807043 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e51f9e52588e4008adf20f443c367b02', 'request_id': 'req-ebffd47e-2355-4a3a-8e46-ee7f64807043'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:54.296509 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ebffd47e-2355-4a3a-8e46-ee7f64807043 admin admin] The token cache is being invalidate because project e51f9e52588e4008adf20f443c367b02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:33:54.296788 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ebffd47e-2355-4a3a-8e46-ee7f64807043 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:54.297448 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-ebffd47e-2355-4a3a-8e46-ee7f64807043 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:54.299009 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1391/2807] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:33:54 2022] DELETE /identity/v3/projects/e51f9e52588e4008adf20f443c367b02 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:33:54.304002 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527904b-1372-45e7-9c2b-ade3bd32596d None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:54.304293 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527904b-1372-45e7-9c2b-ade3bd32596d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:54.304573 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527904b-1372-45e7-9c2b-ade3bd32596d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:54.325895 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-0527904b-1372-45e7-9c2b-ade3bd32596d None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:33:54.327521 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-0527904b-1372-45e7-9c2b-ade3bd32596d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:54.389682 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1417/2808] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:33:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:33:54.398340 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40c88b-c6e0-4eac-bd7d-13f373ec187b None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:54.398629 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40c88b-c6e0-4eac-bd7d-13f373ec187b None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:54.398903 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40c88b-c6e0-4eac-bd7d-13f373ec187b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:54.418895 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-9f40c88b-c6e0-4eac-bd7d-13f373ec187b None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:33:54.420466 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9f40c88b-c6e0-4eac-bd7d-13f373ec187b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:54.428406 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1392/2809] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:33:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:33:54.432782 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0527904b-1372-45e7-9c2b-ade3bd32596d None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:54.442982 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0527904b-1372-45e7-9c2b-ade3bd32596d None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:54.447472 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cea6c1-8f35-4d32-9a90-92298ecb5871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:54.448075 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cea6c1-8f35-4d32-9a90-92298ecb5871 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:54.448350 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cea6c1-8f35-4d32-9a90-92298ecb5871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:54.448617 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cea6c1-8f35-4d32-9a90-92298ecb5871 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:54.449575 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cea6c1-8f35-4d32-9a90-92298ecb5871 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:33:54.449874 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cea6c1-8f35-4d32-9a90-92298ecb5871 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:54.452347 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cea6c1-8f35-4d32-9a90-92298ecb5871 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:54.464060 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1418/2810] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:33:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:33:54.468722 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f40c88b-c6e0-4eac-bd7d-13f373ec187b None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:54.477675 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f40c88b-c6e0-4eac-bd7d-13f373ec187b None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:54.482911 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb287f-d263-49b0-a7bd-51f5080287bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:54.483510 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb287f-d263-49b0-a7bd-51f5080287bf admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:54.483788 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb287f-d263-49b0-a7bd-51f5080287bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:54.484057 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb287f-d263-49b0-a7bd-51f5080287bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:54.484711 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb287f-d263-49b0-a7bd-51f5080287bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:54.486627 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb287f-d263-49b0-a7bd-51f5080287bf admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:54.513947 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1393/2811] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:33:54 2022] POST /identity/v3/projects => generated 389 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:33:54.518775 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cea6c1-8f35-4d32-9a90-92298ecb5871 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:54.529757 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cea6c1-8f35-4d32-9a90-92298ecb5871 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:54.533399 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:33:54.588080 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43bec1a-e8ba-4b9c-b3ca-a910989e967e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:54.588695 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43bec1a-e8ba-4b9c-b3ca-a910989e967e admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:54.588973 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43bec1a-e8ba-4b9c-b3ca-a910989e967e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:54.589242 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43bec1a-e8ba-4b9c-b3ca-a910989e967e admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:54.589907 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43bec1a-e8ba-4b9c-b3ca-a910989e967e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:54.592531 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43bec1a-e8ba-4b9c-b3ca-a910989e967e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:54.612904 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1419/2812] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:33:54 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:33:54.617069 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb287f-d263-49b0-a7bd-51f5080287bf admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:54.627976 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb287f-d263-49b0-a7bd-51f5080287bf admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:54.632585 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7725d78-ab5c-476a-8ce9-0cb2a03707b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:54.633186 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7725d78-ab5c-476a-8ce9-0cb2a03707b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:54.633467 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7725d78-ab5c-476a-8ce9-0cb2a03707b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:54.633740 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7725d78-ab5c-476a-8ce9-0cb2a03707b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:54.634475 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7725d78-ab5c-476a-8ce9-0cb2a03707b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:54.636070 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7725d78-ab5c-476a-8ce9-0cb2a03707b6 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:54.647120 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1394/2813] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:33:54.651578 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43bec1a-e8ba-4b9c-b3ca-a910989e967e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:54.662851 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43bec1a-e8ba-4b9c-b3ca-a910989e967e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:54.667405 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7001fc-111f-404b-9611-54629bf2a35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:54.668008 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7001fc-111f-404b-9611-54629bf2a35c admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:54.668289 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7001fc-111f-404b-9611-54629bf2a35c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:54.668560 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7001fc-111f-404b-9611-54629bf2a35c admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:54.669388 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7001fc-111f-404b-9611-54629bf2a35c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:54.671029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7001fc-111f-404b-9611-54629bf2a35c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:54.681701 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1420/2814] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:33:54.686150 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7725d78-ab5c-476a-8ce9-0cb2a03707b6 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:54.696377 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7725d78-ab5c-476a-8ce9-0cb2a03707b6 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:54.701791 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7541c5-d993-4a38-979c-72e7aa1e44ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:54.702498 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7541c5-d993-4a38-979c-72e7aa1e44ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:54.702778 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7541c5-d993-4a38-979c-72e7aa1e44ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:54.703045 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7541c5-d993-4a38-979c-72e7aa1e44ca admin admin] PATH_INFO: `/v3/projects/9ae2fca706d6439aa8e0b7202a2d3c8c/users/370a9c7475db460a9d4625110b2bc7f0/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:54.717670 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7541c5-d993-4a38-979c-72e7aa1e44ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ae2fca706d6439aa8e0b7202a2d3c8c, user_id=370a9c7475db460a9d4625110b2bc7f0, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:54.720376 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7541c5-d993-4a38-979c-72e7aa1e44ca admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:54.734364 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1395/2815] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:33:54 2022] PUT /identity/v3/projects/9ae2fca706d6439aa8e0b7202a2d3c8c/users/370a9c7475db460a9d4625110b2bc7f0/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:33:54.744825 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c197a-80fa-48a0-a161-33cdf39d2187 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:54.745141 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c197a-80fa-48a0-a161-33cdf39d2187 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:54.745461 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c197a-80fa-48a0-a161-33cdf39d2187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:54.773539 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-991c197a-80fa-48a0-a161-33cdf39d2187 None None] MFA Rules not processed for user `370a9c7475db460a9d4625110b2bc7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:33:54.774814 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-991c197a-80fa-48a0-a161-33cdf39d2187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:54.838120 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1421/2816] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:33:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:33:54.843360 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7541c5-d993-4a38-979c-72e7aa1e44ca admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:54.873294 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:33:54.906527 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7541c5-d993-4a38-979c-72e7aa1e44ca admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:54.911190 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b63f66-3f17-4ec4-94c8-a9f641431b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:54.911807 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b63f66-3f17-4ec4-94c8-a9f641431b48 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:54.912091 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b63f66-3f17-4ec4-94c8-a9f641431b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:54.912366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b63f66-3f17-4ec4-94c8-a9f641431b48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:54.913035 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b63f66-3f17-4ec4-94c8-a9f641431b48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:54.914971 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b63f66-3f17-4ec4-94c8-a9f641431b48 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:54.947062 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1396/2817] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:33:54 2022] POST /identity/v3/projects => generated 389 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:33:54.948660 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c197a-80fa-48a0-a161-33cdf39d2187 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:54.960211 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c197a-80fa-48a0-a161-33cdf39d2187 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:55.020053 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d278bc9-809e-4a30-bd89-f43df27e4cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:55.020685 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d278bc9-809e-4a30-bd89-f43df27e4cac admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:55.020969 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d278bc9-809e-4a30-bd89-f43df27e4cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:55.021247 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d278bc9-809e-4a30-bd89-f43df27e4cac admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:55.021923 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d278bc9-809e-4a30-bd89-f43df27e4cac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:55.023879 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d278bc9-809e-4a30-bd89-f43df27e4cac admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:55.047522 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1422/2818] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:33:54 2022] POST /identity/v3/users => generated 301 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:33:55.051628 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b63f66-3f17-4ec4-94c8-a9f641431b48 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:55.062042 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b63f66-3f17-4ec4-94c8-a9f641431b48 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:55.066684 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a32bbe-1059-4efd-bed9-7e3134c38063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:55.067298 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a32bbe-1059-4efd-bed9-7e3134c38063 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:55.067587 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a32bbe-1059-4efd-bed9-7e3134c38063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:55.067863 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a32bbe-1059-4efd-bed9-7e3134c38063 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:55.068562 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a32bbe-1059-4efd-bed9-7e3134c38063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:55.070251 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a32bbe-1059-4efd-bed9-7e3134c38063 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:55.081159 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1397/2819] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:33:55.085145 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d278bc9-809e-4a30-bd89-f43df27e4cac admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:55.091963 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:33:55.095711 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d278bc9-809e-4a30-bd89-f43df27e4cac admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:55.101369 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a33f4-e994-4a0b-a3e9-16c86c40c225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:55.102122 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a33f4-e994-4a0b-a3e9-16c86c40c225 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:55.102430 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a33f4-e994-4a0b-a3e9-16c86c40c225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:55.102708 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a33f4-e994-4a0b-a3e9-16c86c40c225 admin admin] PATH_INFO: `/v3/projects/b9a879723eaf46519fb4d3dfd9e72ff2/users/6c90824e88644cdeb9b7e3712825cf1f/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:55.117695 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405a33f4-e994-4a0b-a3e9-16c86c40c225 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9a879723eaf46519fb4d3dfd9e72ff2, user_id=6c90824e88644cdeb9b7e3712825cf1f, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:55.120364 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405a33f4-e994-4a0b-a3e9-16c86c40c225 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:55.133587 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1423/2820] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:33:55 2022] PUT /identity/v3/projects/b9a879723eaf46519fb4d3dfd9e72ff2/users/6c90824e88644cdeb9b7e3712825cf1f/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:33:55.138179 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a32bbe-1059-4efd-bed9-7e3134c38063 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:55.148129 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a32bbe-1059-4efd-bed9-7e3134c38063 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:55.204395 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d29741-c514-448e-9ed9-0a0ef97bdc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:55.205019 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d29741-c514-448e-9ed9-0a0ef97bdc5f admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:55.205308 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d29741-c514-448e-9ed9-0a0ef97bdc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:55.205585 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d29741-c514-448e-9ed9-0a0ef97bdc5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:55.206321 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d29741-c514-448e-9ed9-0a0ef97bdc5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:55.207960 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d29741-c514-448e-9ed9-0a0ef97bdc5f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:55.217710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1398/2821] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:33:55.221789 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a33f4-e994-4a0b-a3e9-16c86c40c225 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:55.232612 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a33f4-e994-4a0b-a3e9-16c86c40c225 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:55.237274 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cbec03-6d89-4d04-8cbc-b0e46b48f3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:55.237882 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cbec03-6d89-4d04-8cbc-b0e46b48f3ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:55.238188 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cbec03-6d89-4d04-8cbc-b0e46b48f3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:55.238467 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cbec03-6d89-4d04-8cbc-b0e46b48f3ef admin admin] PATH_INFO: `/v3/projects/b9a879723eaf46519fb4d3dfd9e72ff2/users/6c90824e88644cdeb9b7e3712825cf1f/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:55.240464 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cbec03-6d89-4d04-8cbc-b0e46b48f3ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9a879723eaf46519fb4d3dfd9e72ff2, user_id=6c90824e88644cdeb9b7e3712825cf1f, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:55.243130 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cbec03-6d89-4d04-8cbc-b0e46b48f3ef admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:55.253898 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1424/2822] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:33:55 2022] PUT /identity/v3/projects/b9a879723eaf46519fb4d3dfd9e72ff2/users/6c90824e88644cdeb9b7e3712825cf1f/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:33:55.265502 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b258a-5c51-4af3-9e81-d3dde788047d None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:55.266000 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b258a-5c51-4af3-9e81-d3dde788047d None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:55.266302 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b258a-5c51-4af3-9e81-d3dde788047d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:55.294950 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-ce5b258a-5c51-4af3-9e81-d3dde788047d None None] MFA Rules not processed for user `6c90824e88644cdeb9b7e3712825cf1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:33:55.296171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-ce5b258a-5c51-4af3-9e81-d3dde788047d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:55.365244 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1399/2823] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:33:55 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 04:33:55.375912 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cbec03-6d89-4d04-8cbc-b0e46b48f3ef admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:55.377612 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-03cbec03-6d89-4d04-8cbc-b0e46b48f3ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:55.434073 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cbec03-6d89-4d04-8cbc-b0e46b48f3ef admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:55.437810 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f4b5a-c4ec-4560-babe-31b11aa650ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:55.438447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f4b5a-c4ec-4560-babe-31b11aa650ab service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:55.438726 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f4b5a-c4ec-4560-babe-31b11aa650ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:55.439003 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f4b5a-c4ec-4560-babe-31b11aa650ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:55.447836 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538f4b5a-c4ec-4560-babe-31b11aa650ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:55.449519 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538f4b5a-c4ec-4560-babe-31b11aa650ab service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:55.461482 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1425/2824] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:33:55 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:33:55.595720 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b258a-5c51-4af3-9e81-d3dde788047d None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:55.620248 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:33:55.657590 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b258a-5c51-4af3-9e81-d3dde788047d None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:55.662124 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f9150-6522-4fcb-ab27-154261029856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:55.662765 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f9150-6522-4fcb-ab27-154261029856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:55.663047 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f9150-6522-4fcb-ab27-154261029856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:55.663321 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f9150-6522-4fcb-ab27-154261029856 admin admin] PATH_INFO: `/v3/users/370a9c7475db460a9d4625110b2bc7f0` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:55.664486 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797f9150-6522-4fcb-ab27-154261029856 admin admin] RBAC: Authorizing `identity:delete_user(user_id=370a9c7475db460a9d4625110b2bc7f0)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:55.666468 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797f9150-6522-4fcb-ab27-154261029856 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:55.710355 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-797f9150-6522-4fcb-ab27-154261029856 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '370a9c7475db460a9d4625110b2bc7f0'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:55.711009 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-797f9150-6522-4fcb-ab27-154261029856 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '370a9c7475db460a9d4625110b2bc7f0'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:55.718533 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-797f9150-6522-4fcb-ab27-154261029856 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '370a9c7475db460a9d4625110b2bc7f0'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:55.730243 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-797f9150-6522-4fcb-ab27-154261029856 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '370a9c7475db460a9d4625110b2bc7f0'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:55.737764 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-797f9150-6522-4fcb-ab27-154261029856 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '370a9c7475db460a9d4625110b2bc7f0'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:55.743684 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1400/2825] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:33:55 2022] DELETE /identity/v3/users/370a9c7475db460a9d4625110b2bc7f0 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:33:55.748430 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea92969-1847-46c5-af8d-63e27a35a4c8 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:55.748718 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea92969-1847-46c5-af8d-63e27a35a4c8 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:55.748991 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea92969-1847-46c5-af8d-63e27a35a4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:55.767437 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-7ea92969-1847-46c5-af8d-63e27a35a4c8 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:33:55.768618 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7ea92969-1847-46c5-af8d-63e27a35a4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:55.827568 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1426/2826] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:33:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:33:55.837348 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f9150-6522-4fcb-ab27-154261029856 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:55.838736 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-797f9150-6522-4fcb-ab27-154261029856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:55.895956 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f9150-6522-4fcb-ab27-154261029856 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:55.899698 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb448f9-a9e1-4167-975a-124b8d5b3d98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:55.900316 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb448f9-a9e1-4167-975a-124b8d5b3d98 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:55.900599 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb448f9-a9e1-4167-975a-124b8d5b3d98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:55.900867 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb448f9-a9e1-4167-975a-124b8d5b3d98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:55.910898 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb448f9-a9e1-4167-975a-124b8d5b3d98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:55.912383 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb448f9-a9e1-4167-975a-124b8d5b3d98 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:55.922693 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1401/2827] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:33:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:33:56.189305 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea92969-1847-46c5-af8d-63e27a35a4c8 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:56.190979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7ea92969-1847-46c5-af8d-63e27a35a4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:56.214171 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea92969-1847-46c5-af8d-63e27a35a4c8 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:56.221844 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c54ada-e81c-4bb5-8198-6c254c84f716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:56.222653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c54ada-e81c-4bb5-8198-6c254c84f716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:56.222852 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c54ada-e81c-4bb5-8198-6c254c84f716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:56.223140 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c54ada-e81c-4bb5-8198-6c254c84f716 admin admin] PATH_INFO: `/v3/projects/9ae2fca706d6439aa8e0b7202a2d3c8c` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:56.224331 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c54ada-e81c-4bb5-8198-6c254c84f716 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ae2fca706d6439aa8e0b7202a2d3c8c)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:56.226315 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c54ada-e81c-4bb5-8198-6c254c84f716 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:56.274536 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f6c54ada-e81c-4bb5-8198-6c254c84f716 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ae2fca706d6439aa8e0b7202a2d3c8c', 'request_id': 'req-f6c54ada-e81c-4bb5-8198-6c254c84f716'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:56.280634 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f6c54ada-e81c-4bb5-8198-6c254c84f716 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ae2fca706d6439aa8e0b7202a2d3c8c', 'request_id': 'req-f6c54ada-e81c-4bb5-8198-6c254c84f716'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:56.286184 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f6c54ada-e81c-4bb5-8198-6c254c84f716 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ae2fca706d6439aa8e0b7202a2d3c8c', 'request_id': 'req-f6c54ada-e81c-4bb5-8198-6c254c84f716'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:56.288637 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:33:56.300308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f6c54ada-e81c-4bb5-8198-6c254c84f716 admin admin] The token cache is being invalidate because project 9ae2fca706d6439aa8e0b7202a2d3c8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:33:56.300604 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f6c54ada-e81c-4bb5-8198-6c254c84f716 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:56.301229 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f6c54ada-e81c-4bb5-8198-6c254c84f716 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:56.302788 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1427/2828] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:33:56 2022] DELETE /identity/v3/projects/9ae2fca706d6439aa8e0b7202a2d3c8c => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:33:56.306855 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb448f9-a9e1-4167-975a-124b8d5b3d98 service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:56.317382 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb448f9-a9e1-4167-975a-124b8d5b3d98 service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:56.359890 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:33:56.375974 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd57818-c61c-4746-9075-8eae57cfcd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:56.376620 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd57818-c61c-4746-9075-8eae57cfcd69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:56.376904 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd57818-c61c-4746-9075-8eae57cfcd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:56.377181 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd57818-c61c-4746-9075-8eae57cfcd69 admin admin] PATH_INFO: `/v3/users/6c90824e88644cdeb9b7e3712825cf1f` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:56.378385 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd57818-c61c-4746-9075-8eae57cfcd69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c90824e88644cdeb9b7e3712825cf1f)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:56.380306 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd57818-c61c-4746-9075-8eae57cfcd69 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:56.426685 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3cd57818-c61c-4746-9075-8eae57cfcd69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c90824e88644cdeb9b7e3712825cf1f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:56.427341 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3cd57818-c61c-4746-9075-8eae57cfcd69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c90824e88644cdeb9b7e3712825cf1f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:56.435387 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3cd57818-c61c-4746-9075-8eae57cfcd69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c90824e88644cdeb9b7e3712825cf1f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:56.448298 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3cd57818-c61c-4746-9075-8eae57cfcd69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c90824e88644cdeb9b7e3712825cf1f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:56.456131 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-3cd57818-c61c-4746-9075-8eae57cfcd69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c90824e88644cdeb9b7e3712825cf1f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:56.462494 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1402/2829] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:33:56 2022] DELETE /identity/v3/users/6c90824e88644cdeb9b7e3712825cf1f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:33:56.730385 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c54ada-e81c-4bb5-8198-6c254c84f716 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:56.732215 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f6c54ada-e81c-4bb5-8198-6c254c84f716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:56.744992 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c54ada-e81c-4bb5-8198-6c254c84f716 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:56.804512 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf4404-7fd6-4523-9bb2-ba690329fab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:56.805216 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf4404-7fd6-4523-9bb2-ba690329fab1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:56.805440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf4404-7fd6-4523-9bb2-ba690329fab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:56.805710 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf4404-7fd6-4523-9bb2-ba690329fab1 admin admin] PATH_INFO: `/v3/projects/b9a879723eaf46519fb4d3dfd9e72ff2` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:56.806920 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bf4404-7fd6-4523-9bb2-ba690329fab1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9a879723eaf46519fb4d3dfd9e72ff2)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:56.808834 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bf4404-7fd6-4523-9bb2-ba690329fab1 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:56.854413 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-08bf4404-7fd6-4523-9bb2-ba690329fab1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9a879723eaf46519fb4d3dfd9e72ff2', 'request_id': 'req-08bf4404-7fd6-4523-9bb2-ba690329fab1'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:56.860653 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-08bf4404-7fd6-4523-9bb2-ba690329fab1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9a879723eaf46519fb4d3dfd9e72ff2', 'request_id': 'req-08bf4404-7fd6-4523-9bb2-ba690329fab1'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:56.866140 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-08bf4404-7fd6-4523-9bb2-ba690329fab1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9a879723eaf46519fb4d3dfd9e72ff2', 'request_id': 'req-08bf4404-7fd6-4523-9bb2-ba690329fab1'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:56.877997 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-08bf4404-7fd6-4523-9bb2-ba690329fab1 admin admin] The token cache is being invalidate because project b9a879723eaf46519fb4d3dfd9e72ff2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:33:56.878296 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-08bf4404-7fd6-4523-9bb2-ba690329fab1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:56.878911 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-08bf4404-7fd6-4523-9bb2-ba690329fab1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:56.880399 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1428/2830] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:33:56 2022] DELETE /identity/v3/projects/b9a879723eaf46519fb4d3dfd9e72ff2 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:33:56.885228 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c0f03-cbe6-4b5c-a650-3b207614fa75 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:56.885515 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c0f03-cbe6-4b5c-a650-3b207614fa75 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:56.885791 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c0f03-cbe6-4b5c-a650-3b207614fa75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:56.906080 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-850c0f03-cbe6-4b5c-a650-3b207614fa75 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:33:56.907271 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-850c0f03-cbe6-4b5c-a650-3b207614fa75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:56.969616 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1403/2831] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:33:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:33:57.031308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02331322-0b55-45d2-b00f-7badcfd6f24d None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:57.031589 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02331322-0b55-45d2-b00f-7badcfd6f24d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:57.031863 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02331322-0b55-45d2-b00f-7badcfd6f24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:57.051665 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-02331322-0b55-45d2-b00f-7badcfd6f24d None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:33:57.052856 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-02331322-0b55-45d2-b00f-7badcfd6f24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:57.061280 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1429/2832] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:33:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:33:57.065338 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c0f03-cbe6-4b5c-a650-3b207614fa75 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:57.075065 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c0f03-cbe6-4b5c-a650-3b207614fa75 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:57.079540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a9d90-a7f2-4043-8191-e52681b611f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:57.080143 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a9d90-a7f2-4043-8191-e52681b611f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:57.080421 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a9d90-a7f2-4043-8191-e52681b611f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:57.080694 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a9d90-a7f2-4043-8191-e52681b611f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:57.081386 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2a9d90-a7f2-4043-8191-e52681b611f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80240) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:33:57.081687 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2a9d90-a7f2-4043-8191-e52681b611f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:57.083308 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2a9d90-a7f2-4043-8191-e52681b611f6 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:57.095295 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1404/2833] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:33:57.099325 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02331322-0b55-45d2-b00f-7badcfd6f24d None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:57.109791 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02331322-0b55-45d2-b00f-7badcfd6f24d None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:57.114434 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09686f34-0539-41f0-b5b0-b2bde39482a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:57.115037 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09686f34-0539-41f0-b5b0-b2bde39482a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:57.115314 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09686f34-0539-41f0-b5b0-b2bde39482a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:57.115584 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09686f34-0539-41f0-b5b0-b2bde39482a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:57.116252 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09686f34-0539-41f0-b5b0-b2bde39482a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:57.118151 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09686f34-0539-41f0-b5b0-b2bde39482a2 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:57.144300 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1430/2834] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:33:57 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:33:57.148349 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a9d90-a7f2-4043-8191-e52681b611f6 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:57.159192 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a9d90-a7f2-4043-8191-e52681b611f6 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:57.205802 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:33:57.217983 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835c5dd-d45a-4369-9465-46037f74637c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:57.218606 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835c5dd-d45a-4369-9465-46037f74637c admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:57.218887 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835c5dd-d45a-4369-9465-46037f74637c admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:57.219158 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835c5dd-d45a-4369-9465-46037f74637c admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:57.219834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835c5dd-d45a-4369-9465-46037f74637c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:57.221758 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835c5dd-d45a-4369-9465-46037f74637c admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:57.245362 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1405/2835] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:33:57 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:33:57.246979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09686f34-0539-41f0-b5b0-b2bde39482a2 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:57.257800 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09686f34-0539-41f0-b5b0-b2bde39482a2 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:57.262404 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d9b84-bd34-4d3a-8743-d5a23ef44f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:57.263002 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d9b84-bd34-4d3a-8743-d5a23ef44f82 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:57.263278 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d9b84-bd34-4d3a-8743-d5a23ef44f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:57.263551 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d9b84-bd34-4d3a-8743-d5a23ef44f82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:57.264242 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d9b84-bd34-4d3a-8743-d5a23ef44f82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:57.265825 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d9b84-bd34-4d3a-8743-d5a23ef44f82 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:57.275571 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1431/2836] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:33:57.279979 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835c5dd-d45a-4369-9465-46037f74637c admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:57.289894 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835c5dd-d45a-4369-9465-46037f74637c admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:57.295203 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ff712-89a3-41d6-ad85-7f6e352e9516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:57.295808 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ff712-89a3-41d6-ad85-7f6e352e9516 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:57.296094 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ff712-89a3-41d6-ad85-7f6e352e9516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:57.296374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ff712-89a3-41d6-ad85-7f6e352e9516 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:57.297070 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ff712-89a3-41d6-ad85-7f6e352e9516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:57.298716 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ff712-89a3-41d6-ad85-7f6e352e9516 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:57.309135 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1406/2837] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:33:57.313760 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d9b84-bd34-4d3a-8743-d5a23ef44f82 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:57.323806 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d9b84-bd34-4d3a-8743-d5a23ef44f82 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:57.329525 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbdd4ee-89b4-46f7-ad44-080df247ae61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:57.330224 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdd4ee-89b4-46f7-ad44-080df247ae61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:57.330507 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdd4ee-89b4-46f7-ad44-080df247ae61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:57.330785 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdd4ee-89b4-46f7-ad44-080df247ae61 admin admin] PATH_INFO: `/v3/projects/f9469da735e145b0a88477868608455d/users/8a9a62d1046f456997434918a78c24cc/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:57.345962 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbdd4ee-89b4-46f7-ad44-080df247ae61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9469da735e145b0a88477868608455d, user_id=8a9a62d1046f456997434918a78c24cc, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:57.348664 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbdd4ee-89b4-46f7-ad44-080df247ae61 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:57.361075 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1432/2838] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:33:57 2022] PUT /identity/v3/projects/f9469da735e145b0a88477868608455d/users/8a9a62d1046f456997434918a78c24cc/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:33:57.371003 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba0bf0-5ab8-4b96-ae76-e2f787d4b8eb None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:57.371322 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba0bf0-5ab8-4b96-ae76-e2f787d4b8eb None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:57.371597 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba0bf0-5ab8-4b96-ae76-e2f787d4b8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:57.400287 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-86ba0bf0-5ab8-4b96-ae76-e2f787d4b8eb None None] MFA Rules not processed for user `8a9a62d1046f456997434918a78c24cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:33:57.401488 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-86ba0bf0-5ab8-4b96-ae76-e2f787d4b8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:57.467239 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1407/2839] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:33:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:33:57.471949 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbdd4ee-89b4-46f7-ad44-080df247ae61 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:57.492612 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:33:57.534752 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbdd4ee-89b4-46f7-ad44-080df247ae61 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:57.539419 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c50a4e-a88f-498d-ab7a-e117ab2ff38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:57.540029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c50a4e-a88f-498d-ab7a-e117ab2ff38f admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:57.540308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c50a4e-a88f-498d-ab7a-e117ab2ff38f admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:57.540583 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c50a4e-a88f-498d-ab7a-e117ab2ff38f admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:57.541251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c50a4e-a88f-498d-ab7a-e117ab2ff38f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:57.543179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c50a4e-a88f-498d-ab7a-e117ab2ff38f admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:57.568721 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1433/2840] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:33:57 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:33:57.572714 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba0bf0-5ab8-4b96-ae76-e2f787d4b8eb None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:57.584299 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba0bf0-5ab8-4b96-ae76-e2f787d4b8eb None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:57.644815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86325f6a-db14-435f-8c02-710ad2f84056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:57.645617 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86325f6a-db14-435f-8c02-710ad2f84056 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:57.645893 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86325f6a-db14-435f-8c02-710ad2f84056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:57.646192 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86325f6a-db14-435f-8c02-710ad2f84056 admin admin] PATH_INFO: `/v3/users` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:57.646854 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86325f6a-db14-435f-8c02-710ad2f84056 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:57.648838 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86325f6a-db14-435f-8c02-710ad2f84056 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:57.670136 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1408/2841] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:33:57 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:33:57.674085 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c50a4e-a88f-498d-ab7a-e117ab2ff38f admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:57.684458 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c50a4e-a88f-498d-ab7a-e117ab2ff38f admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:57.689185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd6ca42-8337-48b8-a728-02c0d68df7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:57.689797 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6ca42-8337-48b8-a728-02c0d68df7b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:57.690128 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6ca42-8337-48b8-a728-02c0d68df7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:57.690404 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6ca42-8337-48b8-a728-02c0d68df7b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:57.691107 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd6ca42-8337-48b8-a728-02c0d68df7b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:57.692688 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd6ca42-8337-48b8-a728-02c0d68df7b2 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:57.704343 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1434/2842] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:33:57.708710 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86325f6a-db14-435f-8c02-710ad2f84056 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:57.720010 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86325f6a-db14-435f-8c02-710ad2f84056 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:57.724653 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05087ac0-df1b-4332-ae3a-62434392e871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:57.725319 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05087ac0-df1b-4332-ae3a-62434392e871 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:57.725615 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05087ac0-df1b-4332-ae3a-62434392e871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:57.725876 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05087ac0-df1b-4332-ae3a-62434392e871 admin admin] PATH_INFO: `/v3/projects/455035e8234a46f4ba75ac568a335453/users/2e2fea315a004c658b4f01593023789e/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:57.740571 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05087ac0-df1b-4332-ae3a-62434392e871 admin admin] RBAC: Authorizing `identity:create_grant(project_id=455035e8234a46f4ba75ac568a335453, user_id=2e2fea315a004c658b4f01593023789e, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:57.743277 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05087ac0-df1b-4332-ae3a-62434392e871 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:57.756545 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1409/2843] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:33:57 2022] PUT /identity/v3/projects/455035e8234a46f4ba75ac568a335453/users/2e2fea315a004c658b4f01593023789e/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:33:57.760541 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd6ca42-8337-48b8-a728-02c0d68df7b2 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:57.770942 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd6ca42-8337-48b8-a728-02c0d68df7b2 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:57.829051 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634bd838-187d-4725-b081-5a25ce572e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:57.829601 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bd838-187d-4725-b081-5a25ce572e48 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:57.829885 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bd838-187d-4725-b081-5a25ce572e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:57.830195 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bd838-187d-4725-b081-5a25ce572e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:57.830893 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634bd838-187d-4725-b081-5a25ce572e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:57.832498 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634bd838-187d-4725-b081-5a25ce572e48 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:57.842288 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1435/2844] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:33:57.846417 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05087ac0-df1b-4332-ae3a-62434392e871 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:57.856456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05087ac0-df1b-4332-ae3a-62434392e871 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:57.861084 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362ffab-2e44-4703-b26b-a38175e4978a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:57.861740 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362ffab-2e44-4703-b26b-a38175e4978a admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:57.862054 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362ffab-2e44-4703-b26b-a38175e4978a admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:57.862331 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362ffab-2e44-4703-b26b-a38175e4978a admin admin] PATH_INFO: `/v3/projects/455035e8234a46f4ba75ac568a335453/users/2e2fea315a004c658b4f01593023789e/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:57.864271 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362ffab-2e44-4703-b26b-a38175e4978a admin admin] RBAC: Authorizing `identity:create_grant(project_id=455035e8234a46f4ba75ac568a335453, user_id=2e2fea315a004c658b4f01593023789e, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:57.866940 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362ffab-2e44-4703-b26b-a38175e4978a admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:57.877481 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1410/2845] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:33:57 2022] PUT /identity/v3/projects/455035e8234a46f4ba75ac568a335453/users/2e2fea315a004c658b4f01593023789e/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:33:57.890283 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253a30a-3e41-4e66-8b72-d0ab946405f5 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:57.890567 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253a30a-3e41-4e66-8b72-d0ab946405f5 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:57.890863 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253a30a-3e41-4e66-8b72-d0ab946405f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:57.918208 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-d253a30a-3e41-4e66-8b72-d0ab946405f5 None None] MFA Rules not processed for user `2e2fea315a004c658b4f01593023789e`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:33:57.919490 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d253a30a-3e41-4e66-8b72-d0ab946405f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:57.999544 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1436/2846] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:33:57 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:33:58.010779 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362ffab-2e44-4703-b26b-a38175e4978a admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:58.012541 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-3362ffab-2e44-4703-b26b-a38175e4978a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:58.026215 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:33:58.070205 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362ffab-2e44-4703-b26b-a38175e4978a admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:58.073783 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d67a50d-acf0-4936-a809-0ea1754e4f99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:58.074675 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67a50d-acf0-4936-a809-0ea1754e4f99 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:58.075012 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67a50d-acf0-4936-a809-0ea1754e4f99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:58.075844 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67a50d-acf0-4936-a809-0ea1754e4f99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:58.085009 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d67a50d-acf0-4936-a809-0ea1754e4f99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:58.086505 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d67a50d-acf0-4936-a809-0ea1754e4f99 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:58.096106 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1411/2847] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:33:58 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:33:58.642098 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d253a30a-3e41-4e66-8b72-d0ab946405f5 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:58.658915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:33:58.704526 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d253a30a-3e41-4e66-8b72-d0ab946405f5 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:58.709058 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d334853-732d-4060-a45a-6f24a30298d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:58.709692 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d334853-732d-4060-a45a-6f24a30298d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:58.709994 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d334853-732d-4060-a45a-6f24a30298d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:58.710270 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d334853-732d-4060-a45a-6f24a30298d5 admin admin] PATH_INFO: `/v3/users/8a9a62d1046f456997434918a78c24cc` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:58.711445 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d334853-732d-4060-a45a-6f24a30298d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a9a62d1046f456997434918a78c24cc)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:58.713342 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d334853-732d-4060-a45a-6f24a30298d5 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:58.757356 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3d334853-732d-4060-a45a-6f24a30298d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a9a62d1046f456997434918a78c24cc'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:58.758019 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3d334853-732d-4060-a45a-6f24a30298d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a9a62d1046f456997434918a78c24cc'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:58.758626 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3d334853-732d-4060-a45a-6f24a30298d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a9a62d1046f456997434918a78c24cc'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:58.771952 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3d334853-732d-4060-a45a-6f24a30298d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a9a62d1046f456997434918a78c24cc'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:58.779419 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3d334853-732d-4060-a45a-6f24a30298d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a9a62d1046f456997434918a78c24cc'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:58.792507 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1437/2848] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:33:58 2022] DELETE /identity/v3/users/8a9a62d1046f456997434918a78c24cc => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:33:58.797025 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdceaf-3505-45e4-a41a-47f1b62e5933 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:58.797540 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdceaf-3505-45e4-a41a-47f1b62e5933 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:58.797654 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdceaf-3505-45e4-a41a-47f1b62e5933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:58.816733 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-e2bdceaf-3505-45e4-a41a-47f1b62e5933 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:33:58.817914 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e2bdceaf-3505-45e4-a41a-47f1b62e5933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:58.877646 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1412/2849] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:33:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:33:58.887149 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d334853-732d-4060-a45a-6f24a30298d5 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:58.888601 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-3d334853-732d-4060-a45a-6f24a30298d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:58.948577 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d334853-732d-4060-a45a-6f24a30298d5 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:58.953306 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7aab5-6aac-4542-bbbf-82c1bad4bc58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:58.953921 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7aab5-6aac-4542-bbbf-82c1bad4bc58 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:58.954251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7aab5-6aac-4542-bbbf-82c1bad4bc58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:58.954542 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7aab5-6aac-4542-bbbf-82c1bad4bc58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:58.963528 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b7aab5-6aac-4542-bbbf-82c1bad4bc58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:58.964946 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b7aab5-6aac-4542-bbbf-82c1bad4bc58 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:58.975227 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1438/2850] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:33:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:33:59.251203 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bdceaf-3505-45e4-a41a-47f1b62e5933 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:59.252368 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-e2bdceaf-3505-45e4-a41a-47f1b62e5933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:59.267818 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bdceaf-3505-45e4-a41a-47f1b62e5933 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:59.272458 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a00c7-793f-496a-be82-145acca051b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:59.274134 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a00c7-793f-496a-be82-145acca051b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:59.274419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a00c7-793f-496a-be82-145acca051b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:59.274692 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a00c7-793f-496a-be82-145acca051b0 admin admin] PATH_INFO: `/v3/projects/f9469da735e145b0a88477868608455d` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:59.276121 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161a00c7-793f-496a-be82-145acca051b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9469da735e145b0a88477868608455d)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:59.278794 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161a00c7-793f-496a-be82-145acca051b0 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:59.326719 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-161a00c7-793f-496a-be82-145acca051b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9469da735e145b0a88477868608455d', 'request_id': 'req-161a00c7-793f-496a-be82-145acca051b0'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:59.332836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-161a00c7-793f-496a-be82-145acca051b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9469da735e145b0a88477868608455d', 'request_id': 'req-161a00c7-793f-496a-be82-145acca051b0'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:59.338893 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-161a00c7-793f-496a-be82-145acca051b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9469da735e145b0a88477868608455d', 'request_id': 'req-161a00c7-793f-496a-be82-145acca051b0'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:59.350856 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-161a00c7-793f-496a-be82-145acca051b0 admin admin] The token cache is being invalidate because project f9469da735e145b0a88477868608455d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:33:59.351141 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-161a00c7-793f-496a-be82-145acca051b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:59.351858 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-161a00c7-793f-496a-be82-145acca051b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:59.353585 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1413/2851] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:33:59 2022] DELETE /identity/v3/projects/f9469da735e145b0a88477868608455d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:33:59.357629 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7aab5-6aac-4542-bbbf-82c1bad4bc58 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:59.366988 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7aab5-6aac-4542-bbbf-82c1bad4bc58 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:59.400887 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:33:59.424814 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1dacf-6fb9-4b0b-a549-afc9ae8860b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:59.425444 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1dacf-6fb9-4b0b-a549-afc9ae8860b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:59.425723 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1dacf-6fb9-4b0b-a549-afc9ae8860b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:59.426019 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1dacf-6fb9-4b0b-a549-afc9ae8860b3 admin admin] PATH_INFO: `/v3/users/2e2fea315a004c658b4f01593023789e` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:59.427164 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d1dacf-6fb9-4b0b-a549-afc9ae8860b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e2fea315a004c658b4f01593023789e)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:59.429061 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d1dacf-6fb9-4b0b-a549-afc9ae8860b3 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:59.477391 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d9d1dacf-6fb9-4b0b-a549-afc9ae8860b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e2fea315a004c658b4f01593023789e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:59.478057 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d9d1dacf-6fb9-4b0b-a549-afc9ae8860b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e2fea315a004c658b4f01593023789e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:59.478661 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d9d1dacf-6fb9-4b0b-a549-afc9ae8860b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e2fea315a004c658b4f01593023789e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:59.490359 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d9d1dacf-6fb9-4b0b-a549-afc9ae8860b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e2fea315a004c658b4f01593023789e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:59.497707 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d9d1dacf-6fb9-4b0b-a549-afc9ae8860b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e2fea315a004c658b4f01593023789e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:59.509424 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1439/2852] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:33:59 2022] DELETE /identity/v3/users/2e2fea315a004c658b4f01593023789e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:33:59.781272 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a00c7-793f-496a-be82-145acca051b0 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:33:59.782977 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-161a00c7-793f-496a-be82-145acca051b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:33:59.799376 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a00c7-793f-496a-be82-145acca051b0 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:33:59.834841 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:33:59.859430 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393f147-d7a6-4b01-8353-71ccc05b434f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:33:59.860109 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393f147-d7a6-4b01-8353-71ccc05b434f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:59.860388 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393f147-d7a6-4b01-8353-71ccc05b434f admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:59.860659 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393f147-d7a6-4b01-8353-71ccc05b434f admin admin] PATH_INFO: `/v3/projects/455035e8234a46f4ba75ac568a335453` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:59.861813 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393f147-d7a6-4b01-8353-71ccc05b434f admin admin] RBAC: Authorizing `identity:delete_project(project_id=455035e8234a46f4ba75ac568a335453)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:33:59.863807 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393f147-d7a6-4b01-8353-71ccc05b434f admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:33:59.909743 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f393f147-d7a6-4b01-8353-71ccc05b434f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '455035e8234a46f4ba75ac568a335453', 'request_id': 'req-f393f147-d7a6-4b01-8353-71ccc05b434f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:59.915756 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f393f147-d7a6-4b01-8353-71ccc05b434f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '455035e8234a46f4ba75ac568a335453', 'request_id': 'req-f393f147-d7a6-4b01-8353-71ccc05b434f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:59.922808 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f393f147-d7a6-4b01-8353-71ccc05b434f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '455035e8234a46f4ba75ac568a335453', 'request_id': 'req-f393f147-d7a6-4b01-8353-71ccc05b434f'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:59.937792 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f393f147-d7a6-4b01-8353-71ccc05b434f admin admin] The token cache is being invalidate because project 455035e8234a46f4ba75ac568a335453 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:33:59.938140 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f393f147-d7a6-4b01-8353-71ccc05b434f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:59.938834 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-f393f147-d7a6-4b01-8353-71ccc05b434f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:33:59.941886 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1414/2853] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:33:59 2022] DELETE /identity/v3/projects/455035e8234a46f4ba75ac568a335453 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:33:59.949071 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34b44e-08ec-4def-8996-8e6586ed0387 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:33:59.949879 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34b44e-08ec-4def-8996-8e6586ed0387 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:33:59.950449 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34b44e-08ec-4def-8996-8e6586ed0387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:33:59.970908 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-ed34b44e-08ec-4def-8996-8e6586ed0387 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:33:59.972129 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-ed34b44e-08ec-4def-8996-8e6586ed0387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:34:00.031190 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1440/2854] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:33:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:34:00.039080 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d0b7b-50a2-4bb4-b2ea-f51dd49951f9 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:00.039374 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d0b7b-50a2-4bb4-b2ea-f51dd49951f9 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:00.039683 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d0b7b-50a2-4bb4-b2ea-f51dd49951f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:00.062470 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-312d0b7b-50a2-4bb4-b2ea-f51dd49951f9 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:34:00.063707 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-312d0b7b-50a2-4bb4-b2ea-f51dd49951f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:34:00.071757 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1415/2855] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:34:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:34:00.076612 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34b44e-08ec-4def-8996-8e6586ed0387 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:34:00.086555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34b44e-08ec-4def-8996-8e6586ed0387 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:34:00.091651 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc2b03-63d1-43c7-8294-1563480330d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:34:00.092267 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc2b03-63d1-43c7-8294-1563480330d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:00.092552 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc2b03-63d1-43c7-8294-1563480330d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:00.092835 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc2b03-63d1-43c7-8294-1563480330d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:00.093549 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc2b03-63d1-43c7-8294-1563480330d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:34:00.093853 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc2b03-63d1-43c7-8294-1563480330d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:34:00.095504 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc2b03-63d1-43c7-8294-1563480330d1 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:34:00.110767 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1441/2856] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:34:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:34:00.115408 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d0b7b-50a2-4bb4-b2ea-f51dd49951f9 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:34:00.129840 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d0b7b-50a2-4bb4-b2ea-f51dd49951f9 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:34:00.136504 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced52b4-3f52-4d22-87cc-a1a471360027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:34:00.137534 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced52b4-3f52-4d22-87cc-a1a471360027 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:00.138041 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced52b4-3f52-4d22-87cc-a1a471360027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:00.138508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced52b4-3f52-4d22-87cc-a1a471360027 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:00.139639 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dced52b4-3f52-4d22-87cc-a1a471360027 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:34:00.143059 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dced52b4-3f52-4d22-87cc-a1a471360027 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:34:00.175353 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1416/2857] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:34:00 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:34:00.177826 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc2b03-63d1-43c7-8294-1563480330d1 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:34:00.188948 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc2b03-63d1-43c7-8294-1563480330d1 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:34:00.233866 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:34:00.251675 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69959d20-09df-49c9-b8c6-6c65b2c428ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:34:00.252360 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69959d20-09df-49c9-b8c6-6c65b2c428ec admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:00.252643 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69959d20-09df-49c9-b8c6-6c65b2c428ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:00.252915 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69959d20-09df-49c9-b8c6-6c65b2c428ec admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:00.253596 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69959d20-09df-49c9-b8c6-6c65b2c428ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:34:00.255593 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69959d20-09df-49c9-b8c6-6c65b2c428ec admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:34:00.286223 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1442/2858] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:34:00 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:34:00.290509 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced52b4-3f52-4d22-87cc-a1a471360027 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:34:00.312004 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced52b4-3f52-4d22-87cc-a1a471360027 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:34:00.317119 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03102f26-a6b9-4c76-9d6a-ce3449842a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:34:00.317795 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03102f26-a6b9-4c76-9d6a-ce3449842a24 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:00.318107 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03102f26-a6b9-4c76-9d6a-ce3449842a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:00.318389 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03102f26-a6b9-4c76-9d6a-ce3449842a24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:00.319106 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03102f26-a6b9-4c76-9d6a-ce3449842a24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:34:00.320843 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03102f26-a6b9-4c76-9d6a-ce3449842a24 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:34:00.332116 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1417/2859] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:34:00.336469 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69959d20-09df-49c9-b8c6-6c65b2c428ec admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:34:00.351906 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69959d20-09df-49c9-b8c6-6c65b2c428ec admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:34:00.354206 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed1e44b-8a49-4dde-bc1d-04b03d2c92e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:34:00.354923 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed1e44b-8a49-4dde-bc1d-04b03d2c92e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:00.355202 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed1e44b-8a49-4dde-bc1d-04b03d2c92e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:00.355473 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed1e44b-8a49-4dde-bc1d-04b03d2c92e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:00.356838 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed1e44b-8a49-4dde-bc1d-04b03d2c92e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:34:00.358588 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed1e44b-8a49-4dde-bc1d-04b03d2c92e2 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:34:00.371159 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1443/2860] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:34:00.375926 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03102f26-a6b9-4c76-9d6a-ce3449842a24 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:34:00.386161 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03102f26-a6b9-4c76-9d6a-ce3449842a24 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:34:00.391647 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac9f7c-bcf4-40f3-b294-505f29e2bb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:34:00.392322 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac9f7c-bcf4-40f3-b294-505f29e2bb00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:00.392607 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac9f7c-bcf4-40f3-b294-505f29e2bb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:00.392924 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac9f7c-bcf4-40f3-b294-505f29e2bb00 admin admin] PATH_INFO: `/v3/projects/127bb97d162c49189de0461303fa81bd/users/8a9a52ce728a4c349d940b1b7599d976/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:00.409552 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac9f7c-bcf4-40f3-b294-505f29e2bb00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=127bb97d162c49189de0461303fa81bd, user_id=8a9a52ce728a4c349d940b1b7599d976, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:34:00.412418 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac9f7c-bcf4-40f3-b294-505f29e2bb00 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:34:00.427429 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1418/2861] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:34:00 2022] PUT /identity/v3/projects/127bb97d162c49189de0461303fa81bd/users/8a9a52ce728a4c349d940b1b7599d976/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:34:00.439046 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7e719-e59f-4d57-b564-88dae5827001 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:00.439331 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7e719-e59f-4d57-b564-88dae5827001 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:00.439606 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7e719-e59f-4d57-b564-88dae5827001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:00.470655 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-60c7e719-e59f-4d57-b564-88dae5827001 None None] MFA Rules not processed for user `8a9a52ce728a4c349d940b1b7599d976`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:34:00.471876 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-60c7e719-e59f-4d57-b564-88dae5827001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:34:00.538485 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1444/2862] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:34:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:34:00.550811 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac9f7c-bcf4-40f3-b294-505f29e2bb00 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:34:00.552850 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c1ac9f7c-bcf4-40f3-b294-505f29e2bb00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:34:00.626450 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac9f7c-bcf4-40f3-b294-505f29e2bb00 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:34:00.630171 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277d2f4-e2ee-49c3-9291-e6ba1bfa6017 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '134ef452d7814c73b7c371a1cdbcbc00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:34:00.630796 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277d2f4-e2ee-49c3-9291-e6ba1bfa6017 service cinder] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:00.631080 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277d2f4-e2ee-49c3-9291-e6ba1bfa6017 service cinder] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:00.631349 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277d2f4-e2ee-49c3-9291-e6ba1bfa6017 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:00.640942 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c277d2f4-e2ee-49c3-9291-e6ba1bfa6017 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:34:00.642420 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c277d2f4-e2ee-49c3-9291-e6ba1bfa6017 service cinder] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:34:00.652366 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1419/2863] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:34:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:34:00.680381 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7e719-e59f-4d57-b564-88dae5827001 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:34:00.748604 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7e719-e59f-4d57-b564-88dae5827001 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:34:00.753308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e9390a-4267-488c-b77d-579bfb21c836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:34:00.753983 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9390a-4267-488c-b77d-579bfb21c836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:00.754282 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9390a-4267-488c-b77d-579bfb21c836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:00.754562 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9390a-4267-488c-b77d-579bfb21c836 admin admin] PATH_INFO: `/v3/users/8a9a52ce728a4c349d940b1b7599d976` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:00.755762 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e9390a-4267-488c-b77d-579bfb21c836 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a9a52ce728a4c349d940b1b7599d976)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:34:00.757697 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e9390a-4267-488c-b77d-579bfb21c836 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:34:00.815937 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-08e9390a-4267-488c-b77d-579bfb21c836 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a9a52ce728a4c349d940b1b7599d976'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:34:00.816607 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-08e9390a-4267-488c-b77d-579bfb21c836 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a9a52ce728a4c349d940b1b7599d976'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:34:00.817210 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-08e9390a-4267-488c-b77d-579bfb21c836 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a9a52ce728a4c349d940b1b7599d976'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:34:00.822337 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:34:00.831851 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-08e9390a-4267-488c-b77d-579bfb21c836 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a9a52ce728a4c349d940b1b7599d976'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:34:00.838950 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-08e9390a-4267-488c-b77d-579bfb21c836 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a9a52ce728a4c349d940b1b7599d976'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:34:00.851676 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1445/2864] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:34:00 2022] DELETE /identity/v3/users/8a9a52ce728a4c349d940b1b7599d976 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:34:00.856474 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da68ba-1b7b-4923-943f-29e5ea599c17 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:00.856985 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da68ba-1b7b-4923-943f-29e5ea599c17 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:00.857101 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da68ba-1b7b-4923-943f-29e5ea599c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:00.875118 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:34:00.882161 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-c7da68ba-1b7b-4923-943f-29e5ea599c17 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:34:00.883867 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c7da68ba-1b7b-4923-943f-29e5ea599c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:34:00.942659 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1420/2865] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:34:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:34:00.953352 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e9390a-4267-488c-b77d-579bfb21c836 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:34:00.954766 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-08e9390a-4267-488c-b77d-579bfb21c836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:34:01.011249 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e9390a-4267-488c-b77d-579bfb21c836 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:34:01.015979 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da2d68-1010-4b30-a440-ec41aedc45cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:34:01.016588 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da2d68-1010-4b30-a440-ec41aedc45cc service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:01.016875 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da2d68-1010-4b30-a440-ec41aedc45cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:01.017157 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da2d68-1010-4b30-a440-ec41aedc45cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:01.026348 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da2d68-1010-4b30-a440-ec41aedc45cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:34:01.027774 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da2d68-1010-4b30-a440-ec41aedc45cc service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:34:01.037799 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1446/2866] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:34:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:34:01.305041 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7da68ba-1b7b-4923-943f-29e5ea599c17 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:34:01.306508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-c7da68ba-1b7b-4923-943f-29e5ea599c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:34:01.319254 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7da68ba-1b7b-4923-943f-29e5ea599c17 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:34:01.323914 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ab17d-afef-4311-aba0-46f4c3c48040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:34:01.325878 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ab17d-afef-4311-aba0-46f4c3c48040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:01.325878 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ab17d-afef-4311-aba0-46f4c3c48040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:01.326149 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ab17d-afef-4311-aba0-46f4c3c48040 admin admin] PATH_INFO: `/v3/projects/127bb97d162c49189de0461303fa81bd` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:01.327124 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ab17d-afef-4311-aba0-46f4c3c48040 admin admin] RBAC: Authorizing `identity:delete_project(project_id=127bb97d162c49189de0461303fa81bd)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:34:01.329215 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ab17d-afef-4311-aba0-46f4c3c48040 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:34:01.376950 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-fc8ab17d-afef-4311-aba0-46f4c3c48040 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '127bb97d162c49189de0461303fa81bd', 'request_id': 'req-fc8ab17d-afef-4311-aba0-46f4c3c48040'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:34:01.383290 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-fc8ab17d-afef-4311-aba0-46f4c3c48040 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '127bb97d162c49189de0461303fa81bd', 'request_id': 'req-fc8ab17d-afef-4311-aba0-46f4c3c48040'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:34:01.389110 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-fc8ab17d-afef-4311-aba0-46f4c3c48040 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '127bb97d162c49189de0461303fa81bd', 'request_id': 'req-fc8ab17d-afef-4311-aba0-46f4c3c48040'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:34:01.401749 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-fc8ab17d-afef-4311-aba0-46f4c3c48040 admin admin] The token cache is being invalidate because project 127bb97d162c49189de0461303fa81bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:34:01.402069 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-fc8ab17d-afef-4311-aba0-46f4c3c48040 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:34:01.402703 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-fc8ab17d-afef-4311-aba0-46f4c3c48040 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:34:01.404310 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1421/2867] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:34:01 2022] DELETE /identity/v3/projects/127bb97d162c49189de0461303fa81bd => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:34:01.409288 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb0474-c724-4920-ba4b-516e5f682035 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:01.409578 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb0474-c724-4920-ba4b-516e5f682035 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:01.409860 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb0474-c724-4920-ba4b-516e5f682035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:01.430758 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-4dcb0474-c724-4920-ba4b-516e5f682035 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:34:01.431966 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-4dcb0474-c724-4920-ba4b-516e5f682035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:34:01.492879 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1447/2868] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:34:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:34:01.502776 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde901b4-6edd-49be-b926-7a4493d6c1b3 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:01.503064 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde901b4-6edd-49be-b926-7a4493d6c1b3 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:01.503363 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde901b4-6edd-49be-b926-7a4493d6c1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:01.523242 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-dde901b4-6edd-49be-b926-7a4493d6c1b3 None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:34:01.524450 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-dde901b4-6edd-49be-b926-7a4493d6c1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:34:01.532416 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1422/2869] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:34:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:34:01.536623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb0474-c724-4920-ba4b-516e5f682035 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:34:01.546829 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb0474-c724-4920-ba4b-516e5f682035 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:34:01.551404 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200dc40-2ac4-48a4-b316-f0625421c763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:34:01.552007 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200dc40-2ac4-48a4-b316-f0625421c763 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:01.552287 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200dc40-2ac4-48a4-b316-f0625421c763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:01.552557 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200dc40-2ac4-48a4-b316-f0625421c763 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:01.553251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200dc40-2ac4-48a4-b316-f0625421c763 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 04:34:01.553554 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200dc40-2ac4-48a4-b316-f0625421c763 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:34:01.555174 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200dc40-2ac4-48a4-b316-f0625421c763 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:34:01.567041 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1448/2870] 10.222.0.46 () {58 vars in 1153 bytes} [Wed Jul 27 04:34:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 04:34:01.571281 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde901b4-6edd-49be-b926-7a4493d6c1b3 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:34:01.580914 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde901b4-6edd-49be-b926-7a4493d6c1b3 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:34:01.585598 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7289b894-23ea-4790-bbe7-8c274a90cfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:34:01.586239 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289b894-23ea-4790-bbe7-8c274a90cfa6 admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:01.586523 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289b894-23ea-4790-bbe7-8c274a90cfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:01.586799 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289b894-23ea-4790-bbe7-8c274a90cfa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:01.587473 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7289b894-23ea-4790-bbe7-8c274a90cfa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:34:01.589401 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7289b894-23ea-4790-bbe7-8c274a90cfa6 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:34:01.617313 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1423/2871] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:34:01 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:34:01.621592 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200dc40-2ac4-48a4-b316-f0625421c763 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:34:01.632016 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200dc40-2ac4-48a4-b316-f0625421c763 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:34:01.687833 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d7836-1d00-4c64-af77-9c3eb0928912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:34:01.688464 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d7836-1d00-4c64-af77-9c3eb0928912 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:01.688748 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d7836-1d00-4c64-af77-9c3eb0928912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:01.689024 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d7836-1d00-4c64-af77-9c3eb0928912 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:01.689701 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682d7836-1d00-4c64-af77-9c3eb0928912 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:34:01.691662 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682d7836-1d00-4c64-af77-9c3eb0928912 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:34:01.701460 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:34:01.715092 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1449/2872] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:34:01 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 04:34:01.719175 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7289b894-23ea-4790-bbe7-8c274a90cfa6 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:34:01.729186 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7289b894-23ea-4790-bbe7-8c274a90cfa6 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:34:01.733959 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdf5d7-68e4-4148-a4eb-24e17b212c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:34:01.734587 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdf5d7-68e4-4148-a4eb-24e17b212c25 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:01.734868 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdf5d7-68e4-4148-a4eb-24e17b212c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:01.735143 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdf5d7-68e4-4148-a4eb-24e17b212c25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:01.735843 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bdf5d7-68e4-4148-a4eb-24e17b212c25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:34:01.737437 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bdf5d7-68e4-4148-a4eb-24e17b212c25 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:34:01.747211 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1424/2873] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:34:01.751529 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d7836-1d00-4c64-af77-9c3eb0928912 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:34:01.761650 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d7836-1d00-4c64-af77-9c3eb0928912 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:34:01.766963 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9feb7d-345f-4536-8045-2251410477cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:34:01.767569 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9feb7d-345f-4536-8045-2251410477cb admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:01.767852 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9feb7d-345f-4536-8045-2251410477cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:01.768127 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9feb7d-345f-4536-8045-2251410477cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:01.768819 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9feb7d-345f-4536-8045-2251410477cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:34:01.770446 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9feb7d-345f-4536-8045-2251410477cb admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:34:01.781398 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1450/2874] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:34:01.785439 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdf5d7-68e4-4148-a4eb-24e17b212c25 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:34:01.795386 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdf5d7-68e4-4148-a4eb-24e17b212c25 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:34:01.800014 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da18b1-37b1-4024-bf21-b9f4082ed59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:34:01.800673 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da18b1-37b1-4024-bf21-b9f4082ed59b admin admin] REQUEST_METHOD: `PUT` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:01.800982 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da18b1-37b1-4024-bf21-b9f4082ed59b admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:01.801259 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da18b1-37b1-4024-bf21-b9f4082ed59b admin admin] PATH_INFO: `/v3/projects/1241f54e5ce2416783c5a401e3eded2a/users/f92df6bb851642769a0b7d7b92f5973a/roles/7b6f2ade82434ee79175385de17049eb` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:01.816353 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44da18b1-37b1-4024-bf21-b9f4082ed59b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1241f54e5ce2416783c5a401e3eded2a, user_id=f92df6bb851642769a0b7d7b92f5973a, role_id=7b6f2ade82434ee79175385de17049eb)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:34:01.819062 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44da18b1-37b1-4024-bf21-b9f4082ed59b admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:34:01.832489 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1425/2875] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:34:01 2022] PUT /identity/v3/projects/1241f54e5ce2416783c5a401e3eded2a/users/f92df6bb851642769a0b7d7b92f5973a/roles/7b6f2ade82434ee79175385de17049eb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:34:01.837543 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92d9fd-7a1b-4c7d-a9d8-16f14ed23c3d None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:01.837935 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92d9fd-7a1b-4c7d-a9d8-16f14ed23c3d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:01.838236 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92d9fd-7a1b-4c7d-a9d8-16f14ed23c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:01.856841 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-4d92d9fd-7a1b-4c7d-a9d8-16f14ed23c3d None None] MFA Rules not processed for user `32924f9a30fb4e7b835bb29d116319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:34:01.858210 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-4d92d9fd-7a1b-4c7d-a9d8-16f14ed23c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:34:01.920110 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1451/2876] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:34:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:34:01.929642 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da18b1-37b1-4024-bf21-b9f4082ed59b admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:34:01.931577 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-44da18b1-37b1-4024-bf21-b9f4082ed59b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:34:01.962737 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:34:01.991213 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da18b1-37b1-4024-bf21-b9f4082ed59b admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:34:01.994793 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50878b5f-5055-4b00-b599-50bd057b02f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:34:01.995404 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50878b5f-5055-4b00-b599-50bd057b02f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:01.995683 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50878b5f-5055-4b00-b599-50bd057b02f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:01.995953 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50878b5f-5055-4b00-b599-50bd057b02f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:02.004952 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50878b5f-5055-4b00-b599-50bd057b02f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:34:02.006438 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50878b5f-5055-4b00-b599-50bd057b02f1 service neutron] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:34:02.016258 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1426/2877] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:34:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:34:05.975447 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d49087-05bd-4bea-b9c2-c647bc860e8e None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:05.976285 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d49087-05bd-4bea-b9c2-c647bc860e8e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:05.976746 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d49087-05bd-4bea-b9c2-c647bc860e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:06.007996 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-b2d49087-05bd-4bea-b9c2-c647bc860e8e None None] MFA Rules not processed for user `f92df6bb851642769a0b7d7b92f5973a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:34:06.009255 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b2d49087-05bd-4bea-b9c2-c647bc860e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:34:06.078666 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1452/2878] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:34:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:34:06.083085 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50878b5f-5055-4b00-b599-50bd057b02f1 service neutron] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:34:06.093508 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50878b5f-5055-4b00-b599-50bd057b02f1 service neutron] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:34:06.098140 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e5f339-ecc6-452d-8cb4-7b4b9a239e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:34:06.098751 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5f339-ecc6-452d-8cb4-7b4b9a239e9a admin admin] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:06.099033 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5f339-ecc6-452d-8cb4-7b4b9a239e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:06.099304 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5f339-ecc6-452d-8cb4-7b4b9a239e9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:06.099966 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e5f339-ecc6-452d-8cb4-7b4b9a239e9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:34:06.101951 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e5f339-ecc6-452d-8cb4-7b4b9a239e9a admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:34:06.133414 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1427/2879] 10.222.0.46 () {60 vars in 1153 bytes} [Wed Jul 27 04:34:06 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:34:06.137351 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d49087-05bd-4bea-b9c2-c647bc860e8e None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:34:06.148780 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d49087-05bd-4bea-b9c2-c647bc860e8e None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:34:06.215093 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b0a8e-ccb8-497b-b72a-bc8b98b589b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:34:06.215732 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b0a8e-ccb8-497b-b72a-bc8b98b589b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:06.216011 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b0a8e-ccb8-497b-b72a-bc8b98b589b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:06.216917 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b0a8e-ccb8-497b-b72a-bc8b98b589b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:06.217768 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b0a8e-ccb8-497b-b72a-bc8b98b589b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:34:06.219768 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b0a8e-ccb8-497b-b72a-bc8b98b589b9 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:34:06.237478 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:34:06.244220 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1453/2880] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 27 04:34:06 2022] POST /identity/v3/users => generated 298 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 04:34:06.248395 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e5f339-ecc6-452d-8cb4-7b4b9a239e9a admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:34:06.259116 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e5f339-ecc6-452d-8cb4-7b4b9a239e9a admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:34:06.263752 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b741bab-0127-402a-82b3-e3f1a949464b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:34:06.264664 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b741bab-0127-402a-82b3-e3f1a949464b admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:06.264946 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b741bab-0127-402a-82b3-e3f1a949464b admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:06.265216 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b741bab-0127-402a-82b3-e3f1a949464b admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:06.265915 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b741bab-0127-402a-82b3-e3f1a949464b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:34:06.267545 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b741bab-0127-402a-82b3-e3f1a949464b admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:34:06.282815 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1428/2881] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:34:06.283983 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b0a8e-ccb8-497b-b72a-bc8b98b589b9 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:34:06.294179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b0a8e-ccb8-497b-b72a-bc8b98b589b9 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:34:06.299448 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23893279-69bb-420e-92c8-e89adbf248a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:34:06.300124 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23893279-69bb-420e-92c8-e89adbf248a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:06.300433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23893279-69bb-420e-92c8-e89adbf248a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:06.300719 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23893279-69bb-420e-92c8-e89adbf248a6 admin admin] PATH_INFO: `/v3/projects/dbdccc3882474169a602fadc01e79417/users/10ae55591ef344a5b48f9ac3aa86bd27/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:06.316759 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23893279-69bb-420e-92c8-e89adbf248a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbdccc3882474169a602fadc01e79417, user_id=10ae55591ef344a5b48f9ac3aa86bd27, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:34:06.319554 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23893279-69bb-420e-92c8-e89adbf248a6 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:34:06.336475 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1454/2882] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:34:06 2022] PUT /identity/v3/projects/dbdccc3882474169a602fadc01e79417/users/10ae55591ef344a5b48f9ac3aa86bd27/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 04:34:06.341913 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b741bab-0127-402a-82b3-e3f1a949464b admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:34:06.352659 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b741bab-0127-402a-82b3-e3f1a949464b admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:34:06.410422 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8653ab3-7226-4c39-810c-2ddc53c62297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:34:06.411048 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8653ab3-7226-4c39-810c-2ddc53c62297 admin admin] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:06.411331 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8653ab3-7226-4c39-810c-2ddc53c62297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:06.411609 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8653ab3-7226-4c39-810c-2ddc53c62297 admin admin] PATH_INFO: `/v3/roles` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:06.412310 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8653ab3-7226-4c39-810c-2ddc53c62297 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:34:06.413944 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8653ab3-7226-4c39-810c-2ddc53c62297 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:34:06.424379 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1429/2883] 10.222.0.46 () {58 vars in 1122 bytes} [Wed Jul 27 04:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 04:34:06.428585 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23893279-69bb-420e-92c8-e89adbf248a6 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:34:06.438700 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23893279-69bb-420e-92c8-e89adbf248a6 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:34:06.443251 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce26150e-9aee-426b-9885-a6fc96e4210c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:34:06.443903 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce26150e-9aee-426b-9885-a6fc96e4210c admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:06.444179 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce26150e-9aee-426b-9885-a6fc96e4210c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:06.444449 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce26150e-9aee-426b-9885-a6fc96e4210c admin admin] PATH_INFO: `/v3/projects/dbdccc3882474169a602fadc01e79417/users/10ae55591ef344a5b48f9ac3aa86bd27/roles/cf771a8e94ee46149eb081347fea6d67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:06.446403 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce26150e-9aee-426b-9885-a6fc96e4210c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbdccc3882474169a602fadc01e79417, user_id=10ae55591ef344a5b48f9ac3aa86bd27, role_id=cf771a8e94ee46149eb081347fea6d67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:34:06.449008 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce26150e-9aee-426b-9885-a6fc96e4210c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:34:06.460479 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1455/2884] 10.222.0.46 () {60 vars in 1483 bytes} [Wed Jul 27 04:34:06 2022] PUT /identity/v3/projects/dbdccc3882474169a602fadc01e79417/users/10ae55591ef344a5b48f9ac3aa86bd27/roles/cf771a8e94ee46149eb081347fea6d67 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:34:11.026888 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e80dc5-273c-4cac-b849-d6c93ba730f9 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:11.028009 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e80dc5-273c-4cac-b849-d6c93ba730f9 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:11.028489 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e80dc5-273c-4cac-b849-d6c93ba730f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:11.078611 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-a7e80dc5-273c-4cac-b849-d6c93ba730f9 None None] MFA Rules not processed for user `10ae55591ef344a5b48f9ac3aa86bd27`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:34:11.080303 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-a7e80dc5-273c-4cac-b849-d6c93ba730f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:34:11.175419 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1430/2885] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:34:11 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:34:11.184029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce26150e-9aee-426b-9885-a6fc96e4210c admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:34:11.189781 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-ce26150e-9aee-426b-9885-a6fc96e4210c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:34:11.318003 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce26150e-9aee-426b-9885-a6fc96e4210c admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:34:11.323058 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c35ed-f5fe-442e-95d6-855bcfc61b97 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e98a41a0f304fc2bb36a18a77a2d729', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:34:11.323695 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c35ed-f5fe-442e-95d6-855bcfc61b97 service glance] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:11.324740 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c35ed-f5fe-442e-95d6-855bcfc61b97 service glance] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:11.325029 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c35ed-f5fe-442e-95d6-855bcfc61b97 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:11.396986 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15c35ed-f5fe-442e-95d6-855bcfc61b97 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:34:11.399555 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15c35ed-f5fe-442e-95d6-855bcfc61b97 service glance] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:34:11.412424 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1456/2886] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:34:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:34:11.548731 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43ec7d-46fe-4d22-bf61-843faf1fabea None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:11.549022 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43ec7d-46fe-4d22-bf61-843faf1fabea None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:11.549295 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43ec7d-46fe-4d22-bf61-843faf1fabea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:11.570446 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-bd43ec7d-46fe-4d22-bf61-843faf1fabea None None] MFA Rules not processed for user `8ec07ee1efba4e8089b94f24be338f86`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:34:11.571833 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-bd43ec7d-46fe-4d22-bf61-843faf1fabea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:34:11.587688 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:34:11.643098 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1431/2887] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Jul 27 04:34:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:34:11.666916 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c35ed-f5fe-442e-95d6-855bcfc61b97 service glance] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:34:11.668433 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e15c35ed-f5fe-442e-95d6-855bcfc61b97 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:34:11.728973 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:34:11.748332 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c35ed-f5fe-442e-95d6-855bcfc61b97 service glance] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:34:11.753471 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a354c58-06c9-460e-84b9-61406b19039b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57b5fa7517034a39930ce7c9d12e1875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:34:11.754154 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a354c58-06c9-460e-84b9-61406b19039b service swift] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:11.754438 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a354c58-06c9-460e-84b9-61406b19039b service swift] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:11.754743 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a354c58-06c9-460e-84b9-61406b19039b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:11.762811 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a354c58-06c9-460e-84b9-61406b19039b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:34:11.764440 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a354c58-06c9-460e-84b9-61406b19039b service swift] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:34:11.772669 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1457/2888] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 27 04:34:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 04:34:20.181831 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdfb8cf-f819-4774-a995-f22c8fe20193 None None] REQUEST_METHOD: `POST` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:20.182435 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdfb8cf-f819-4774-a995-f22c8fe20193 None None] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:20.182563 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdfb8cf-f819-4774-a995-f22c8fe20193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:20.202142 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.auth.core [None req-9bdfb8cf-f819-4774-a995-f22c8fe20193 None None] MFA Rules not processed for user `f92df6bb851642769a0b7d7b92f5973a`. Rule list: `[]` (Enabled: `True`). {{(pid=80240) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:34:20.203275 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-9bdfb8cf-f819-4774-a995-f22c8fe20193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:34:20.210098 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1432/2889] 10.222.0.46 () {58 vars in 958 bytes} [Wed Jul 27 04:34:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:34:21.769270 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=6 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:34:22.139240 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dce99f-2b66-4f60-8bd5-68455853fd60 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:22.139577 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dce99f-2b66-4f60-8bd5-68455853fd60 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:22.139856 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dce99f-2b66-4f60-8bd5-68455853fd60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:22.160636 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-d3dce99f-2b66-4f60-8bd5-68455853fd60 None None] MFA Rules not processed for user `8ec07ee1efba4e8089b94f24be338f86`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:34:22.161927 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d3dce99f-2b66-4f60-8bd5-68455853fd60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:34:22.175296 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1458/2890] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Jul 27 04:34:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:34:22.184526 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdfb8cf-f819-4774-a995-f22c8fe20193 None None] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:34:22.197204 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdfb8cf-f819-4774-a995-f22c8fe20193 None None] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:34:22.202744 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d13a39-8d36-4068-986c-6fcc6c63c1fe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57b5fa7517034a39930ce7c9d12e1875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:34:22.203507 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d13a39-8d36-4068-986c-6fcc6c63c1fe service swift] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:22.203835 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d13a39-8d36-4068-986c-6fcc6c63c1fe service swift] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:22.204178 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d13a39-8d36-4068-986c-6fcc6c63c1fe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:22.212417 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d13a39-8d36-4068-986c-6fcc6c63c1fe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:34:22.214109 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d13a39-8d36-4068-986c-6fcc6c63c1fe service swift] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:34:22.222535 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1433/2891] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 27 04:34:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 04:34:32.154534 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:34:32.218965 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=26 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:34:48.819528 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dce99f-2b66-4f60-8bd5-68455853fd60 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:34:48.820954 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d3dce99f-2b66-4f60-8bd5-68455853fd60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:34:48.875854 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dce99f-2b66-4f60-8bd5-68455853fd60 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:34:48.879428 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd9c96-a2ca-40ee-871a-464ed62a756c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4847f3736d844d78a206b7eab5f5fcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:34:48.880044 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd9c96-a2ca-40ee-871a-464ed62a756c service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:34:48.880326 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd9c96-a2ca-40ee-871a-464ed62a756c service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:34:48.880601 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd9c96-a2ca-40ee-871a-464ed62a756c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:34:48.889478 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cd9c96-a2ca-40ee-871a-464ed62a756c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:34:48.890933 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cd9c96-a2ca-40ee-871a-464ed62a756c service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:34:48.900168 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1459/2892] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:34:48 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:34:58.896038 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=26 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:36:00.212584 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d13a39-8d36-4068-986c-6fcc6c63c1fe service swift] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:36:00.214069 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-42d13a39-8d36-4068-986c-6fcc6c63c1fe service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:36:00.298945 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d13a39-8d36-4068-986c-6fcc6c63c1fe service swift] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:36:00.304599 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941762bd-4eb9-467f-b037-3d5f08ec13e9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c08fba2d7de42218400c5e652cc3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:36:00.305352 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941762bd-4eb9-467f-b037-3d5f08ec13e9 service placement] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:36:00.305640 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941762bd-4eb9-467f-b037-3d5f08ec13e9 service placement] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:36:00.305942 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941762bd-4eb9-467f-b037-3d5f08ec13e9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:36:00.307964 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-941762bd-4eb9-467f-b037-3d5f08ec13e9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:36:00.320552 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941762bd-4eb9-467f-b037-3d5f08ec13e9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:36:00.322195 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941762bd-4eb9-467f-b037-3d5f08ec13e9 service placement] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:36:00.333475 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1434/2893] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:36:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:36:07.600104 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9ee77-6f91-48df-8694-1b9e0ad26496 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:36:07.600574 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9ee77-6f91-48df-8694-1b9e0ad26496 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:36:07.600679 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9ee77-6f91-48df-8694-1b9e0ad26496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:36:07.620402 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-e0f9ee77-6f91-48df-8694-1b9e0ad26496 None None] MFA Rules not processed for user `8ec07ee1efba4e8089b94f24be338f86`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 04:36:07.621602 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e0f9ee77-6f91-48df-8694-1b9e0ad26496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:36:07.629068 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1460/2894] 10.222.0.46 () {58 vars in 967 bytes} [Wed Jul 27 04:36:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 04:36:07.669564 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941762bd-4eb9-467f-b037-3d5f08ec13e9 service placement] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:36:07.681887 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941762bd-4eb9-467f-b037-3d5f08ec13e9 service placement] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:36:07.686882 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4ec12-ef02-4295-a244-3b31cd34a700 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '57b5fa7517034a39930ce7c9d12e1875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:36:07.687489 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4ec12-ef02-4295-a244-3b31cd34a700 service swift] REQUEST_METHOD: `GET` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:36:07.687773 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4ec12-ef02-4295-a244-3b31cd34a700 service swift] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:36:07.688050 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4ec12-ef02-4295-a244-3b31cd34a700 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:36:07.696472 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b4ec12-ef02-4295-a244-3b31cd34a700 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:36:07.698074 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b4ec12-ef02-4295-a244-3b31cd34a700 service swift] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:36:07.706412 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1435/2895] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 27 04:36:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 04:36:14.051263 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9ee77-6f91-48df-8694-1b9e0ad26496 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:36:14.114762 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9ee77-6f91-48df-8694-1b9e0ad26496 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:36:14.119268 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef84b35-e5d7-47e1-9062-894435af589d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:36:14.119908 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef84b35-e5d7-47e1-9062-894435af589d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:36:14.120185 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef84b35-e5d7-47e1-9062-894435af589d admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:36:14.120458 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef84b35-e5d7-47e1-9062-894435af589d admin admin] PATH_INFO: `/v3/users/f92df6bb851642769a0b7d7b92f5973a` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:36:14.121623 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef84b35-e5d7-47e1-9062-894435af589d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f92df6bb851642769a0b7d7b92f5973a)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:36:14.123559 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef84b35-e5d7-47e1-9062-894435af589d admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:36:14.168760 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-6ef84b35-e5d7-47e1-9062-894435af589d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f92df6bb851642769a0b7d7b92f5973a'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:36:14.169429 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-6ef84b35-e5d7-47e1-9062-894435af589d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f92df6bb851642769a0b7d7b92f5973a'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:36:14.170137 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-6ef84b35-e5d7-47e1-9062-894435af589d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f92df6bb851642769a0b7d7b92f5973a'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:36:14.182621 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-6ef84b35-e5d7-47e1-9062-894435af589d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f92df6bb851642769a0b7d7b92f5973a'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:36:14.189347 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-6ef84b35-e5d7-47e1-9062-894435af589d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f92df6bb851642769a0b7d7b92f5973a'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:36:14.203240 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1461/2896] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:36:14 2022] DELETE /identity/v3/users/f92df6bb851642769a0b7d7b92f5973a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:36:14.382621 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4ec12-ef02-4295-a244-3b31cd34a700 service swift] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:36:14.395647 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4ec12-ef02-4295-a244-3b31cd34a700 service swift] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:36:14.454324 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5fe27f-d724-4f2c-8f00-289123af0ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:36:14.454961 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5fe27f-d724-4f2c-8f00-289123af0ae4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:36:14.455239 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5fe27f-d724-4f2c-8f00-289123af0ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:36:14.455512 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5fe27f-d724-4f2c-8f00-289123af0ae4 admin admin] PATH_INFO: `/v3/projects/1241f54e5ce2416783c5a401e3eded2a` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:36:14.456832 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5fe27f-d724-4f2c-8f00-289123af0ae4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1241f54e5ce2416783c5a401e3eded2a)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:36:14.458785 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5fe27f-d724-4f2c-8f00-289123af0ae4 admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:36:14.505836 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-6e5fe27f-d724-4f2c-8f00-289123af0ae4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1241f54e5ce2416783c5a401e3eded2a', 'request_id': 'req-6e5fe27f-d724-4f2c-8f00-289123af0ae4'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:36:14.512019 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-6e5fe27f-d724-4f2c-8f00-289123af0ae4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1241f54e5ce2416783c5a401e3eded2a', 'request_id': 'req-6e5fe27f-d724-4f2c-8f00-289123af0ae4'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:36:14.517414 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-6e5fe27f-d724-4f2c-8f00-289123af0ae4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1241f54e5ce2416783c5a401e3eded2a', 'request_id': 'req-6e5fe27f-d724-4f2c-8f00-289123af0ae4'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:36:14.530166 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-6e5fe27f-d724-4f2c-8f00-289123af0ae4 admin admin] The token cache is being invalidate because project 1241f54e5ce2416783c5a401e3eded2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:36:14.530456 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-6e5fe27f-d724-4f2c-8f00-289123af0ae4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:36:14.531079 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-6e5fe27f-d724-4f2c-8f00-289123af0ae4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:36:14.532649 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1436/2897] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:36:14 2022] DELETE /identity/v3/projects/1241f54e5ce2416783c5a401e3eded2a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:36:14.536877 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef84b35-e5d7-47e1-9062-894435af589d admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:36:14.538310 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-6ef84b35-e5d7-47e1-9062-894435af589d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:36:14.551991 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef84b35-e5d7-47e1-9062-894435af589d admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:36:14.607308 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a8d33-dd49-4815-a855-a372a6d53267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:36:14.607934 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a8d33-dd49-4815-a855-a372a6d53267 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:36:14.608214 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a8d33-dd49-4815-a855-a372a6d53267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:36:14.608590 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a8d33-dd49-4815-a855-a372a6d53267 admin admin] PATH_INFO: `/v3/users/10ae55591ef344a5b48f9ac3aa86bd27` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:36:14.609744 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a8d33-dd49-4815-a855-a372a6d53267 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10ae55591ef344a5b48f9ac3aa86bd27)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:36:14.611874 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a8d33-dd49-4815-a855-a372a6d53267 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:36:14.625575 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:36:14.658176 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-5c4a8d33-dd49-4815-a855-a372a6d53267 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10ae55591ef344a5b48f9ac3aa86bd27'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:36:14.659182 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-5c4a8d33-dd49-4815-a855-a372a6d53267 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10ae55591ef344a5b48f9ac3aa86bd27'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:36:14.659787 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-5c4a8d33-dd49-4815-a855-a372a6d53267 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10ae55591ef344a5b48f9ac3aa86bd27'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:36:14.671658 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-5c4a8d33-dd49-4815-a855-a372a6d53267 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10ae55591ef344a5b48f9ac3aa86bd27'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:36:14.679106 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-5c4a8d33-dd49-4815-a855-a372a6d53267 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10ae55591ef344a5b48f9ac3aa86bd27'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:36:14.691310 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1462/2898] 10.222.0.46 () {58 vars in 1224 bytes} [Wed Jul 27 04:36:14 2022] DELETE /identity/v3/users/10ae55591ef344a5b48f9ac3aa86bd27 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:36:14.859329 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5fe27f-d724-4f2c-8f00-289123af0ae4 admin admin] Authenticating user token {{(pid=80240) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:36:14.860957 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.fernet_utils [None req-6e5fe27f-d724-4f2c-8f00-289123af0ae4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80240) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:36:14.875459 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5fe27f-d724-4f2c-8f00-289123af0ae4 admin admin] Validating token access rules against request {{(pid=80240) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:36:14.879746 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:36:14.934282 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e831f9b-56e6-4fa5-bc98-b156aab37ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32924f9a30fb4e7b835bb29d116319d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5672fa915b944f15a9f6ebcb29dba655', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80240) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:36:14.934931 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e831f9b-56e6-4fa5-bc98-b156aab37ddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:36:14.935222 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e831f9b-56e6-4fa5-bc98-b156aab37ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:36:14.935475 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e831f9b-56e6-4fa5-bc98-b156aab37ddd admin admin] PATH_INFO: `/v3/projects/dbdccc3882474169a602fadc01e79417` {{(pid=80240) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:36:14.936760 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e831f9b-56e6-4fa5-bc98-b156aab37ddd admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbdccc3882474169a602fadc01e79417)` {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:36:14.938705 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e831f9b-56e6-4fa5-bc98-b156aab37ddd admin admin] RBAC: Authorization granted {{(pid=80240) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:36:14.984965 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-5e831f9b-56e6-4fa5-bc98-b156aab37ddd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbdccc3882474169a602fadc01e79417', 'request_id': 'req-5e831f9b-56e6-4fa5-bc98-b156aab37ddd'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:36:14.991501 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-5e831f9b-56e6-4fa5-bc98-b156aab37ddd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbdccc3882474169a602fadc01e79417', 'request_id': 'req-5e831f9b-56e6-4fa5-bc98-b156aab37ddd'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:36:14.997065 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-5e831f9b-56e6-4fa5-bc98-b156aab37ddd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbdccc3882474169a602fadc01e79417', 'request_id': 'req-5e831f9b-56e6-4fa5-bc98-b156aab37ddd'} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:36:15.010509 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-5e831f9b-56e6-4fa5-bc98-b156aab37ddd admin admin] The token cache is being invalidate because project dbdccc3882474169a602fadc01e79417 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80240) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 04:36:15.010925 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-5e831f9b-56e6-4fa5-bc98-b156aab37ddd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:36:15.011880 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG keystone.notifications [None req-5e831f9b-56e6-4fa5-bc98-b156aab37ddd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80240) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 04:36:15.013721 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: [pid: 80240|app: 0|req: 1437/2899] 10.222.0.46 () {58 vars in 1233 bytes} [Wed Jul 27 04:36:14 2022] DELETE /identity/v3/projects/dbdccc3882474169a602fadc01e79417 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 04:36:24.687778 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=21,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:36:25.006584 dv-cinder-850989-2-iscsi devstack@keystone.service[80240]: DEBUG dbcounter [-] [80240] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80240) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 04:41:06.200390 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a8d33-dd49-4815-a855-a372a6d53267 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 04:41:06.201782 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5c4a8d33-dd49-4815-a855-a372a6d53267 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:41:06.339004 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a8d33-dd49-4815-a855-a372a6d53267 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 04:41:06.397272 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2066c1d9-eefa-4ef0-962b-cee61b9f508b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c08fba2d7de42218400c5e652cc3de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646d320aa5a4f78bfccaf941cf46478', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 04:41:06.397817 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066c1d9-eefa-4ef0-962b-cee61b9f508b service placement] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 04:41:06.398126 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066c1d9-eefa-4ef0-962b-cee61b9f508b service placement] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 04:41:06.399047 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066c1d9-eefa-4ef0-962b-cee61b9f508b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 04:41:06.400950 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2066c1d9-eefa-4ef0-962b-cee61b9f508b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 04:41:06.412595 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066c1d9-eefa-4ef0-962b-cee61b9f508b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 04:41:06.414259 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066c1d9-eefa-4ef0-962b-cee61b9f508b service placement] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 04:41:06.424071 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1463/2900] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 27 04:41:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 04:41:16.419876 dv-cinder-850989-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}